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FINAL REPORT DOCUMEMTATIOM 

VOLUI'IE I EXBCUri'VE SUMMARY ~ "Low Cost Program Practices for Ftitm’s 

NASA Space Programs." 




VOLIME II APPEOTIX - "Low Cost Program Practices for Futiire NASA 

Space Programs - Engineering Memoranda #2 

Volume II includes : 

Engineering. Memoranda as follows: 

LGPP-2. Impact of fflL-D-lOOO Upon Space Programs^ and Proposed Cost 

Improvement of Changes.: A: 

LCPP-3* Potential Cost Benefits Derivable from Changes to Ten (lO) 
Selected Cost Brivj.ng Specifications. 

LCPP-1|. Specification Coiranonality Analysis for Six (6) Selected NASA 
Space Programs. 

LCP?“5. a Plan for Establishing a NASA Central Specification Control 
Office (OSCO) 

LCPP-6. A Low Cost Version of NHB 5300.Ii(lB) (Qualit7/‘ Assurance 
Provisions for Aerospace Contractors) 

LGPP-7. Space Experiments Support Program 71-2; Program History 
and Spacecraft System Descriptions. 

LCPP-B. System Engineering & Integration (SE&l), Cost. Impact Analysis. . 
LCPP-9. Ground Support Equipment (GSE) Cost Impact Analysis. 

LGPP-10. Space Experiment Support Program 71-2; Cost Analysis. 

LCPP-11. Space Experiment Support Program 71-2; Program Practice Analysis 
LC.PP-12, Program Effects Relationships (PER) Survey., y 
LCPP-I3. Specification Analysis; MlL-D-lOOO ( Engrg Dr'awirigs & Lists ) 
LCPP-llj.,, SpeGificatibn Implementation Goimaentary; MIL-D-IGOO , 

LCPP-15. Asse ssrnent of Risk Impact; Low Cost Program Practices. 

LCPP-I6. MIL-^STD-810B, MIL-STD-883A; Critical Review & Commentary. 
LCPP'^17. A New-Look Standardized Work Breakdown Structure (WBS). . 
LCPP-I8 Proposed NASA Policy (implementation) Directives - LCPP * 
LCPP-I9 SOTA Survey j Quantification of Program Risk. : 

LCPP-20 A Low Cost NASA Engineering Drawings. & Lists Specification. 
Submitted to NASA separately. Not included in the Technical Report Voliime 
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FORWORD 

This Technical Report documents the progress and outcome .S- of MASA/HQ Study 
Contract NAS W-2752; an eighteen (l8) months in-depth analysis of NASA 
Program Practices. The study is the second of two conducted by Lockheed Mis- 
siles and Space Company Inc.,* the first was undertaken in compliance with a 
contract let by NASA. MSFC, and represented the initial attempt to quantify 
those practices routinely imposed by MSA upon aerospace contractors, vjhich 
are, or have been, causative of significant increases in NASA Space Program Costs. 
wTiile the first study identified, and to the degree possible, quantified 
NA.S A Program Practicesin the four* program areas of Specifica-tions, DociMentation, 
NASA General Contracting PracticevS, and Program Definition j this second study 
exam.ined those program a.reas identified, but not fully investigated in the 
previous undertaking. To that end, further analysis was conducted in the Pi’ogram 
Specifications area, and an attempt was made to examine the Program Practices 
pursued in the areas of Systems Engineering and Integration (SEM), and Ground 
Support Equipment (GSE), In addition, the study provides a preDAminary exarpple 
of a novel, more, fully definitized Work Breakdown Structure, and a qualitative 
assessment of the Pi’ogram Risks believed to be associated with the application 
of . Low Cost Pr.actices to NASA Space Programs . Both the task, and the 
Risk Assessment arose from discovery during the first study, and the confirmat- 
ion during the early stages of the present study p that: 

(a) Current NA.S A. MBS definition is insufficient to permit t. he detailed 
level of costing necessary to provid,e Program Managers wirh insight 
into exactly how, and where, program dollars are spent,. 

(b) Program managers have, and still do, . express concern over the 
suspected increment in program risks -attendant upon the implem- 
entation of Low Cost Prog.ram Practices, 

During the course of the NASA/MSFC study, a Program Effects Relationship 
Handbook was produced in, preliminary form. One of the tasks, cf this study was 
to determine, by NASA./lndustiy, survey, Program Management reaction to the 
utility and application of such a Handbook, The results of this suz-vey are 
treated in the main body o.f the report. The results/outcom.es of a.11 study tasks, 
have been documentecl by : Engineering i^morauda (EMs) , released as each task was 
completed. .U.1 of these Ms form the text of the Appendix, to this Volume, and 
they appear as they were written, in the , informal- style agreed as adequate by 
NASA. 
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Section 1 
INTRODUCTION 

This study has been performed for NASA/HQ under the auspices and direction 
of the NASA/HQ Low Cost Sirstems Office . In the continuing MSA. drive toward 
effective. Low Cost Space Programs, the Low Cost Systems Office (LGSO) 
monitored' the stiidy conducted under contract to NAS.a/MSFC closely. To a 
considerable degree, the investigation of Program Practices, and the c’-rolution. 
of lower cost alternates for the historical practices, depends upon the 
eocLstence of adequate program records descriptive of historical practices. 

In some cases, for instance the investigation of the Cv>3t & Practices aspects 
of SE&I, and GSS, resulted in the conclusion that data are insufficient to merit 
further inquiry into the subjects. In yet another case, the desire of NASA/HQ 
LCSO to verify Costs and Practices reported by another Government Agency, gave 
rise to a task of this study (Task Tj . The completion of the MSPC funded study 
culminated in the production of a Program. Effects Relationship(s) Handbook, 
heiTce a task of tliis study v;as to deterro3.ne by NA.SA./lndustry Survey fLe degree 
of accepta.nce and utility of such a program estimating tool. Thus, the study 
continues some tasks began in the MSEC study, ampUfiss them, and furnishes 
results. Those tasks new to tliis study, and of specific interest to LCSO 
have been pursued to the degree believed profitable, and where curtailment has 
proved necessary due to lack of data to. continue a given task 3 such a condition 
has been reported to LCSO., IfCO has issued re-direction of effort as necessaxy, 
and as suggested by LMSC. This Technical Report represents the study effort, 
both as originally ema-sioned and as re-directed. . 
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1.1. STUDI BACKGROUND 


This study covers the period June 197U to December 1975 j and as originally 
planned had four (Ii) major tasks: 

Task 1 The compilation of Program/Cost Data Banlcs for the LMSCI spacecraft 

program P 71-2 and other NASA & DoD Space Programs. 

Task 2 . The conduct of Low Cost Program Practices analysis in five (5) areas 
of program technical interest; (1) Specifications, (2) Systems Engin- 
eering & Integration (SE<5cl), (3) Ground Support Eciui.pme'ut (C-SE), (ii) 
Program Work Breal<down Structinres (>JBS), (5) Program Risk versus Low 
Cost Practices assessnient. 

Task 3 : Generation, data of value to NASA, for inclusion in a forthcomingMASA 
issued series of gui.delines and instructions on Lew Cost Program Pract- 
ices (NASA Implementation Directives) 

Task k The conduct of a NASA and Industry Survey to determine, reactions to Low 
Cost Methodology, and to determine . the potential acceptance, and utility 
of a PER Handbook; whose format and content would emphasize the use of 
• Low Cost PERs when planning NASA space programs on a Low Cost basis. 

The background giving rise to these tasks, and the subsequent re-direction of 
the study due to the curtailment of two (2) ’of those tasks is as follows: 
l.lol. Task 1 Background : At the outset of this study (approx. Julj" 1971; ) 

USAE SAMSO issued a series of digests in the aerospace industry techjiical jour- 
nals in which the claim was made that the USAF space vehicle test (STP) 
program Was cost effective/low cost. NASA/HQ LCSO was interested in obtaining 
details of the program practices, employed by USAF which pei’mitted this claim 
to be made> As one of the USAF programs discussed in the: published information 
was the IJ^SC program P-71-2, LCSO requested that this program, be included in 
the study as part of Task 1. There were two reasons for the requs.st/directive ; 

(1) Comparison of methods and costs of USAF programs , vs NASA programs. , 

(2) Low Cost Hardware used foi' the P 71-2 program could be described for 
inclusion in. the iLiSA Catalog of ^Available Standard, Nardware (CASH),, 
which was in the process of eompilation Under LCSO auspices. 

In addition to the, data for the ? 71-2, 'program LCSO requested similar data for 
LMSG programs P-50, and P-95. niSC applied for USAF permission to release 
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data, but dua to the classified nature of both of these programs, the permis- 
sion was deni-ed. However the P 71-2 programmatic data was compiled and 
furnished to the LCSO in the form of three ( 3 ) Ms» M LCPP-7, which dealt 
with the hardware disclosure, was delivered to LGSO separately, and hence is 
not included in this report, nor ax’s Ms.. LCPP-10, the P 71-2 Cost Anal- 
ysis, and LCPP-11, the P 1-2 Low Cost Practices Analysis. A.11 of these Ms 
were dsHvered to LCSO as they were completed, and both #10 & #11 are incliided 
in the Appendix to this report- The overview of the information these two EMs 
contain is treated subsequently in Section ^ following, as is the subject of 
the stixdy Data Bank. 

1,1.2. Background, Ta.sk 2 This task encompassed five (^) areas of technical 
analysis. The first, which was concerned with specifications imposed by NASA 
for space programs, had its origins in the LGSO desire to explore fmther 
the cost driving aspects of specifications, which had been investigated in 
some depth during the course of the earlier MSEC study. The outcomes of this 
portion of task 2 are include d, together with the highlights of the anal-ysis, 
Section 3 of this. report. The second area of analysis was coneex’ned with the 
general category of Systems Integration and the even more loosely defined diricip 
line of System Engineering (SE&l), ’'/Jhile no detailed investigation of this 
subject was undsrtai-cen during the course of the MSEC study, cursor;/- examination 
of the cost category revealed that the costs acquired under this heading varied 
from program to program, from contractor to contractor, and the overall cost 
breakdowns wei’e neither detad.led, nor understandable. LCSO believed that an 
investigation of the SE&I area of interest, would furnish some insight into 
means of effecting cost saxdngs to programs. Even less detailed information 
exists for the GSE aspects of program costing. The usual practice has been to 
lump all GSE costs under a very general category, with no further breakdown as 
tO; costs, or disposition of the CfSE after prograras have been completed. LCSO 
thus approved the MSG recommendation that this area merited in vestiga,t ion. 
Details of the outcomes of these stxxdies are to be found in Sections 6 & 7 
of this report . One of the difficulties besetting both the MSEC study, and 
this study has; been the scarcity of detailed programmatic cost data* Although 
Work Breakdown Structures (WBS) have, been employed as a cost acquisition/cost, 
tracking/reporting tool, they have seldom contained sufficient detailed ta.sk 
data to permit either valid inquiry into what program activity has been under- 
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taken for some given level of activity fiondingj or what detailed work cost 
how much money. If it cannot be demonstrated that program costs per unit task 
are of some given order, then it is not possible to estimate with any accurac 3 /' 
the savings to be derived from the implementation of low cost practices. The 
"New Look WBS " task was proposed and undertaken to afford insight into such 
an area of interest, and the description of the task and its results are con- 
tained in Section 8 of this report, as weU. as EM LCPP-17 , which furnishes 
details, and is contained in the Appendd.:c. The Pro g ram Risk versus Low Cost 
Practices assessment was undertaken as a task arising from NASA/Xndustry concern 
over possible increases in program risk attendant upon the institution of Low 
Cost Program Practices. The concern of both NASA and Industry space program man- 
agers with the potential risks associated with the adoption of Low. Cost Prograu 
practices, was voiced by almost all of the personnel interviewed during the. 
course of Task U, the PER Survey . Chronologically, Task It preceeded the accomplish- 
ment of the 5th of the technical inqxiiriesj however since the concern over risk 
was so widespread as revealed bjr the PER Survey, tkis area of technical interest 
was accorded as detailed a. Qj.ialitat ive treatment, as the availability of data 
on the subject permitted. The . results of the qualitative risk .a.nalysis are 
summariaed in Section 5 of this report , and EM lJCPP-15 > included in the Appendix, 
furnishes details, 

1.1.3. Task 3 Background ; .By agreement with NASA/HQ LC.SO this task consisted 
of the generation of a series : of guidelines, and instructions on the subject 
of Low Cost Methodology in general, and Low Cost Program Practices in particular. 
These were to be used by H/Q in the preparation of Implementation Directives . 
Implementation directives is a generic term for those official documents concerned 
with Low Costs. As such, these directives are e>q^ected to foi’ra the subject matter 
of either Policy Directives, Management Instructions, or both. For this reason 
the EMSC data/guidelines/instructions are examples onl;/, and are not included 
in the Appendix, to this. Technical. Report. An BM, LGPP-lS has been prepared con- 
taini,ng these examples, however, and this EM has been delivered to NASA/KQ sep- \ 
arately. 

1.1. li. Task Ii Background 0 tjhe c*ose of the MSEC prior study, IldSC prepared , 
and delivered to; NAS-A/HQ copies of a preliminary PSR. Handbook. PSRs may be 
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defined as an estimating device for program managers which shows the relation- 
ship of a given program practice,, to program costs. Costs of customary pract- 
ices were contrasted with Low Cost Prograju alternate Practices, In order to 
publicize the existence to this programmatic tool, and to not only improve 
the content of the handbook^ but also gain ideas into what other PERS might 
be of value to NASA and industry program pi aimer Sj and managers j a PER Survey 
was undertaken. The exbent of this survey, the modus operandi for conducting it 
the foimal questionnaire prepared by LMSC for completion by the persons inter- 
viewed, and the outcomes of it, all are summarised in Section h of thj.s report. 
Details of the survey, and an example of the questioimaira form the subject 
matter of EI4 LCPP-12 , which is included in the Appendix. 

1,2. Study Goals ; The major goal of the studsr was to evolve by various means 
alternate program practices for NASA space programs, which, when implemented, 
would effect financial savu-ngs when considered versus customary practices 
which have proved to be cost drivers. The general goal had four subsidiary 
objectives as follows; 

(1) By investigation, determine the cost impact of current NASA practices 
applied to hardware procurement, monitoring/control, develo|mient/qual- 
ification upon program costs at aerospace contractors. Such investig- 
ation was believed to be valuable in the identification of significant 
cOvSt trends, and the determination of overal program cost effects.. 

(2) analysis of space programs historical data, or equivalent data/ 
proposals for near future programs, specific program procedures, or 
contractor-requirements changes causative of program cost, reductions 
could be identified. 

(3) Determination of the cost reduction impact of Lov? Cost Practices, by 
means of historica3. data comparisons, or by a process of logical 
estimation. Such data were to be displayed in a manner designed to 
facilitate cost effective decisions for future space programs, by 
management personnel. 

(I4) Preparation of specimen documents, such as re-written versions of 

selected specifications ., Such documents, styled to accomodate Low 

Cost methodology, and its implementation, could then be imposed by 

NASA on future space programs, with appreciahle cost benefits as the 

result, ^ / 
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1,3. Study Scope ; The span of the study was eighteen months^ from June 197U 
until Decemeber 1975/ and the final report was to be prepared in the last month 
of the study. 

The tasks of the study were in number^ as shown in the task outline appear- 
ing on Page 1-2. Task 2, the analysis of specifications, had five (5) sub- 
tasks which are discussed as to pertinence and backgroinid in Section 1.1 . 

All tasks were completed, and the outcomes were documented in a series of I9 
Engineering Memoranda, LCPP-2 through LCPP-I9. Copies of all of these memos 
were delivered to MASA as they were generated, 'and fui’ther copies of the series 
appear in the Appendix to this technical report,, with two (2) exceptions. 

EM LGPP- 7,'and EM LCPP-I8. These two Ef-Is had no direct bearing on the outcomes 
of the LGPP analysis tasks as such, for reasons stated in Section 1,1. and 
were therefore excluded from the Appendix, 


l,h. Study Guidelines and Assmiptions ; One assumption was made, which pertained 
throughout the study, and six gid.de Ifnes were proposed to .NASA./HQ who concurred 
with these propositions. These are listed as follows; , 

1 .14 .1. Assumption : In all cases 197,5 technolog:/' has been assijmed adequate for 
implementation of improved, cost effective program practices, evolved as outcomes 
of the study. 

1.14.2. Gid-delines ; 

1.14.2.1. Cost impac t areas selected for the study emphasis analyses V 7 ill be 
those mutually agreed potentially profitable in terms of evolution of cost 
effective practicas, by LMSC and NA-Sa/kQ. 

1.14.2.2 . Spacecraft technology changes , will not be considered to influence . 
the basic program practices under investigation. , 

1. 1. 2. 3 . The study will consider only un-manned, automated spacecraft programs. . 
(This guideline was NASA/HQ directed) 

I.I4.2.I4 .Data prepared during the previous MSEC study, to be used for a starting 
point for the current study. 

1.. 14..2.5. Those areas belisvad to possess the highest cost-saaing potential 

will form the am.phasis areas of this study. . 

1.14.2.6.. Data, to facilitate the new analysis areas of ■ AND GSS are to be 

furnished by NASA, as are specification- tree listings. Other data 
available from such .sources as the Library of Gongress , Scientific Data 


to be used, as, and vrhere, pe^t^nsnt. 
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SECTION 2 

SUT4MARY OF STUDY RESULTS 

2.1. Study Results by Task ; Each of the sixteen (l6) EYs included in the 
Appendix to this report furnishes Conclusions and reconmendations concerning 
the subject addr'essed by the EM. Further^ each section of this report sets 
forth the outcomes of the ta.sks perfox’mcj during the study. To facilitate 
understanding of the entire study task as a whole, this: section 2 includes 

a very brief overview of the study tasks ^ and the major conclusions to be 
drawn when considering not only the taslcs themsslves^ but the study as a whole. 

2.1.1. Task -1 Ovem niLeW: This task was undertaken during a period of roughly the 

first eight (8) months of the program,, and was dE/ided into two (2) parts. 

Part 1 consisted of am.assing a complete teciinical description of the space- 
craft and its subsystems. In addition to the technical performance description, 
and data sheets descriptive of the components used, sketches and drawings of the 
major components were compiled from program originals. All of this material was 
compiled into an EN giving the complete program history of the program, and, 

■in two appendices Appendix A, & B, furnished Program Plight Data Reports, and 
Component Data Summaries. The main body of the EM was voluminous, consisting of 
202 handwritten, but fully legible pa.gss,* and Appendices A <?/ B consisted: of 
6 $ and 173 handwritten pages respectively. As NASA wished to obtain the component 
data as soon as possible for inclusion into GASH, and the page count was so large, 
the COR agreed to accept both the EM and the Appendices in handwritten state. 

By further agreement, since the program had received acclaim as Low Cost, and used 
Low Cost Program Practices in part , the EM was delivered early, and does not app- 
ear in the Appendix to this volum-e. Information from the GOR stated that the SM 
was- satisfactory as delivered (March 15*75) - The entire package v?as E-1 TjCPP-7 . 

Part 2 of the analysis of this program P 71-2, covered the Program Costs Analysis, 
and the Progi-am Practices analysis. Two sepa.rate E?'Is were written to cover 
these aspects of the analysis taskj E4 LCPP-10 for the Cost Analysis , issued . 

May l5 1975. and EM LGPP-ll issued on the same date , covering the Program Practices 
Analysis. 

In. brief, the cost analysis v/as compiled from the program summaries, but with the 
additional feature that non-i’ecurring cost .s were segregated from recurring costs, ; 
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for unit and operating cost categories^ and the second feature of importance 
Within the subsystem costs, vSegrogation was carried down to the "black-box'' 
or major assembly level to pro-'/ids as much functional detail as possible. 

The Program Practices Study for P 71-2 showed that high eo^uipment, and design in- 
heritance from one program to another contributes a major cost advantage, and 
contrasted 3 bow Goat Practices in nonrecurring costs vex'sus their complementarj’' 
high cost oracticas. A similar ms.atis of contrasting cost was eraplcyed in the unit 
rec'urring cost area. Disregarding the cost sa-snlngs due to high inheritance, 
the analysis showed at least 215? of Tota_l Program Costs (TPC) could ha.ve been 
saved by implementation of recommended Low Cost Practices. (For details see Appdx) 
2.1.2. Task '2 O/erview ; This task may be considered as the main technical 
analysis portion of the LGPP stiidy. As stated earlier in this report, it consisted, 
of $ technical areas of interest: 

0 Specifications Analysis ... EMs 2,3,l.b^;6,13j l)i, 16, 
o Systems Integration and Engineering (SE&l) Analysis. M 8. 
o Groxxnd Support 'Equipment (GSE) Analysis... M 9 
o Program Work Breakdown Structure (WBS) J^ralysis EM 17 . 
o Program Risk versus Low Cost Practices Assessment ... 15'. 

The specifications anal^Asis carried on throughout the program, with the 
first tasks being completed, and the Ms describing them delix'^ered dm''ing the 
1st 8 months of the program (EMs 2,3,i4^5 d; 6) . At that time a review of the prog- 
ram was made by NASA/ KQ and the decision was taken that further analysis in depth 
should be umdertaken in the areas of MIL- D~ 1000, MIL STD 8103 & MIL-STD-883A 
WhJ.ch cover Engineering Drawings & Lists, En''airoranentaI Test Methods, and 
Test Methods for Microcircuits respectively. MIL-D-1000 was completely re- 
written in a Lovj Cost version, and a draft of the re-write was furnished to 
NASA./kQ, at the occasion of the A.ugust 20 COR ' s visit to LhlSG . An EM ( LCPP-2Q ) was 
written to cover the new version of the specification itself, and E»4s 13, . 

were issii.ed at the end of the 7nonth, the first being details of the analysis which 
contributed to the re -write of the document, and the second being an Ipiplementat- 
ion conunentary. The final specificatior analysis of the two test oriented documents 
was completed during the , month of September, . and deliver;'' of the E?T, EM -16, 
was made during the early part of October 76 • 
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The S7/5tems Engineering & Integration. (SS?cl) Analysis was tindertaksn during 
the early phase of the prograrij except that the EM describing the task was not 
issued until the month of April; approx 9-10 months after the actual contract 
start. As the data search progressed during the first 5-6 months of the contr- 
act, it became apparent that data in a suitable form for cost impact analysis 
did not exist; and that contractor-to~contractor , little commonality of \mder- 
st ending of what the cost category implies appears evinced on past programs. 

For similar reasons: the cost quantification and identification of Low Cost 
Practices was not possible! largely due to the incomplete and inconsistent 
record keeping in the areas of Program Definition, RFP, and >JBS prepax-ation. 

Some determinations of a general nature were possible however, and. these appear 
in greater detasLl in Section 6 of this report, as well- as in the EM LCPP-8 . 
contained within the Appendix, 

The C-SE Anal7/~si5 (GSE) This task was undertaken in parallel with the SE?cI 
analysis. The -fii'st step in the task progression was to prepare a C-SS 
questionnaire for circulation to. NASA field centers and industry. The question- 
naire was distributed to NASA .centers -.and selected companies within the aero- 
space industry as follows: NASA AMES R . C. GSFG . . MSFC . Industry : Boeing Rorp, 
CTaneral Electric Corp; Hughes, Mrcraft Coxp, Jet Propulsion Laboratories,. 
Fairchild-Hiller Corp, RCA Gorp, Philco Corp TOL, and TRW Corp. Of these, 
only Hughes Aircraft Corp, MSFO/lRW, GSFG /Philco, GSFC/P>.CA responded, and the 
response was not ^ full in some cases. To. augment these data,L!!^SC pro’^xided 
questionnaires to both the P 71-2 program office and the P~50 program office . 

The former did not reply, and. the latter responded, The significa-Ut trend 
appeal's to be toward lower GSE costs- and this end is obtained by making maorim- 
UTi possible use of the GSE already in possession of contractors, so that h-igh . 
equipnent inheritance prograra-to-program is maintained. The subject is treated 
in greater detail in Section 7 of this report, and details are contained in . 
full in: the; SM. LCPP-9 issued May 1 ::1975, ^ copy of which is in the Appendix. 

The 7JBS Task Due to. the pressure of continuing the tasks of .specification 
analysis, SE&I and , GSE analysis, and documenting the . outcomes of the PER : 
survey (see Section, h), the start of the VfBS task; wa-s deliberately delayed. 
Further unforseen delays: .were occasioned by an accident injury to the LCPP- 
progr am manager, under whose .leadershiip the task was to. be conductsd: and the 
unavailability of personnel originally scheduled to. perform the- task. ' When -work 
was commenced,: it was. beld^eved . that the LMSC version,: of the GSGSG technique, 

2-3 

LOCKHEED MISSILES 8; SPACE COMPANY, INC. 


GASPER j a computerized cost acquisiiiion and tracking operation validated ly the 
USAF; would be acceptable for use. GASPER provj.de s excellent cost acquisition 
and siuiiming capability for work packages traditionally used for WBS operatiouj 
where the cost computations are additive in nature . As the data search progressed 
and the definition of the WBS task was expanded to include functional disciplines 
and descending levels of cost detail) it becaiue apparent that the overall 
computa.tion, tracking and reporting of all program effort costs was combin - 
atorial in nature . tJhen all possible permutations of the many combinations of 
functional task^ and hardware/software production dollar cost acquisition 
methods wa.s made, the total was of the order of 50j000,* CASPER is not suatab3.e 
to handle ‘the combinatorial aspects of .such a large order of cost allocation 
subsets, as the means for allocating costs to such a large number of chargeable 
areas is impractical, both from a cost-to-acquire, and the human problems inher- 
ent in, logging the data, aspects. Resolutions for these problems, and the final 
scope of the task, together v?ith its outcomes are detailed in Section 8 of this 
report, total detail being contained in EM LGPP-17 included i.n the Appendix. 

The Risk vs Low Cost Practices Assessment Task -This task arose from, the outcome 
of the PER Survey which was LGPP Program Task I 45 detrslls of which are discussed 
in Section Ig of this report, and are ampli.fied fully in EI4 LCPP-12, ,of^ the 
Appendix. One of the major concerns of those personnel from both .KfASA and ind- 
ustry responding the the PER . que stionnaire ; was the increase in program risk 
suspected to result from the implementation of Low Cost Practices. A sur-vey 
of current techniques for assessing risk revealed that these are of a probabil- 
istic nature , . and considerable data of the prior performace ' t;;/pe are required, 
to manipulate and iterate the many mathematical models vJith any, degree of sure- 
ty of outcome. Therefore the approach taicsn to the assessment of programmatic 
risk im.plicit in the application of Low Cost Practices was to perform analyses 
of each:. Program Practice qualitatively . Da.ta sheets were constructed, a sample; 
of which appears; in Section ,9 of this .report . , These Sheets, list each practice, 
show the program area impacted by it, and furnish an assessment qualitatively ' 
ixi terms of High, Medium, and Low, likelihood of an , adverse cost Impact* Risks 
3xe shovjn with respect to the classic affects on Performance, Schedule, and Pro- 
gram Gost;. Finally, a Eelta Risk assessment is given which Shows whether or not 
application of a practice may impart: an Increase, Decrease, : or No ChaUk^ cond- 
itton to, the program areals)- impacted. Outcomes are given in EM' LCPP-lS included 
in the Appendi.x, and the highlights are contained in Section Ii. 
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2.2. Major Conclusions : 

2.2.1. General ; In addition, to the major conclusions listed belov?^ each section 
of this report cax’ries any recommendations necessary to amplify the contents 

of the section. In preparing the several sections of the report, illustrations 
have been included. These illustrations, whether tables, graphs, ox' other : 
pictorial matter, portray the highlights of the data contained in the SMs 
which furnish the details of the study data,. 

2.2.2. Record Keeping ; Throughout both the MS?C prior study, and this current 
study, a major difficulty has been the lack of good historical DETAILED program 
cost vs performance records , NASA-imposed record keeping requirements for 
aerospace contractors are in dire need of major overhaul, if good program 

cost visibilitjr is to be achieved, and maintained, throughout future programs. 

2.2.3. Spocifications : Specifications' can, and do drive costs. However, it is 

not: so much the affect of any one specification, as the interactive, and cum- 
ulative affects of the total specification '’tree'’, and its subordinate hier- 
archy imposed as a routine pra.ctioe.; that contributes to program costs. 

2.2.1t. SEct.1 : V/ithin the aerospace industry and NASA, there is no clear, xini ver- 
sa! definition of what constitutes SS&I program a.cti’xity. Reported costs for 
such a loosely defined program activity vary considerably, and there is no deal 
assurance that NASA has received what was required for the m.oney spent. 

2,2,5^. GSE : GSE is poorly reported, as a cost item, and detailed cost data are 
either unavailable^ non-existent,- or both j program-to-program. However^ from 
the da,ta that are available (and as reported hj those responding to the GSE 
survey) it may be inferred that: 

(a) GSE inheritance from one prograii to another within contractors' 
facilities, can save progi-am costs . 

(b) The inheritance operation has been used with, coat effective,-:: . 

results on such programs asj (JPL-Bbeing Gorp)., 

AE:(GSEC-RCA-.Gorp), P.71-2 (USAF-SAHSO/STP-EMSG>:; : 

2.2 ,,6. Program Risk vs Low Cost Practices ; From the a_ualitative listing of 
program risks associated with Low Co-st practices it would appear that Program 
Manager concern with such risks is largely psychological , rathe r than tangible . 
However a valid method of assessing prograxa risks prior to program outset is 
needed, to promote the adoption; of Low Cost Practices by future -NASA progranis.. . 
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2.2.7* Standard Work Breakdown Structure (TOS) : LI'ISC believes that a standard 
WBS is reqiiired for use with NASA programs. However, there is a very definite 
relationship which exists between the level of cost detail acquired/reported, 
and the complexity, overall cost effectiveness, and practicality of the 
acquistion/iTecording/reporting process. It is evident that the sheer magnitude 
of the cost acquition and work definition-by-package process, may iwell prohibit 
its usej from both an ease of handling, and cost viewpoint. VJhat he optimum 
Standard li'/BS should be, in terms of scope and application, could not be dstermiiisd 
within the constraints of the schedule and finds available for this study. 
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SECTION 3 


SPECIFICATION IMPACT A.NALYSIS 

3,1 Task Introduction : The specifications task for this study is a logical 
continuation from the task of similar ncrnis which was undertaken for MSFC in 
the previous stiidy. Specifications documents, which by definition are those 
documents setting f t ’•’th program requirements other than detailed performance 
paranieterSj tend to drive program costs. Appreciable sa^/ings can be effected 
if the specifications imposed by NASA are simplified, and reduced to only that 
minijnum essential for satisfactor^r program performance/management. Perhaps 
among Low Cost Program practices, specifications imposition is the most simple 
to analyse, and modify! for three reasons; 

(1) Specifications are imposed for all NASA programs, and are 
\d-sible to all program personnel concerned , 

(2) The psychological attitx;da of technical personnel toward 

specifications is that they are a "Necessary and 

thus any improvement in them, or reduction in their number, 
is desirable. 

(3) tkiatever costs are driven by a specification, relate directly 
to the document. If it is not imrok*=?d, the costs it implies 
are not incurred, at least to the degree the specification 
provides, . 

3,2. General Outline : The specifications analysis task was to a large degree 
directed by NASa/hQ . The overall task may be divided into three ( 3 ) se ctions 
as follows: ' ■ ■ ■ 

o Analyses performed during the first 7-8 rnori'ths of the ^ study 
o Analyses completed bjr the mid-terrx review, 

0 Analysis completed subsequent to the mid-term review, at which 
time, some re -direction of the specifications task was given 
by NASA/HQ. 

Those analyses falling within the first ca,tegory may be listed as follows : . 
S'I-2 Impact . of MIL-D-lOQO Upon Space Programs, etc. 

EM-3 Cost Benefits derivable from 10 Selected Specs., 

SJi-Ii Specification CGmmonality for 6 Selected MSA Programs . 
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EM-5 a Plan for a NASA Central Specificatiion Control Office 
EiM-6 a Low Cost Version of NHB 5300. [}(1B) (Quality/- Assurance) 
Specification anaJ.yse.s undertaken and completed within the next work category 
are as follows : 

o- A complete re-write of MIL-D-IOOO, Engineering Drawings and Lists^ 
was made j and copies v/ere handed to the NA.SA/KQ G OR who visited the LMSO facil- 
ity on May 5/1975- This work wa.s NOT made the subject of an EM as the decision 
was mads to issue the specifica,tion as a document ’which could he used by NASA 
as 'wriiten^ if NASA so elected/ Later j EM LCPP-20 number was assigned to the spec 

o -During the visit by the COR, NASA/HQ Reliability/Quality Assur- . 
ance comments to EM-6 were furnished to LMSC. A response to these- comments ■ 
was prepared and mailed to MSA/HQ 10 days later, and after the mid-term 
revi.ew June 18/75.> the EM-6, the pertinent comments by HQ/QA, and the LMSC 
responses to those comments were discussed ’with Mr George khitej the NASA 
reviewing authority for Rel/QA. 


Work in the. third category was undertaken as follows: 

o EM-13 Specification Anal 3 ''sis 14IL-D-1000 ’was issued August 26/75 
o Eiyi-lii Specification Implementation Comjf!enta.ry was issued Aug 28/75' 
Both of these SMs amplified the details of the analysis performed to complete : 
the re-write of the MlL-D-1000 specification, and fur zii. shed also suggested me- 
ans by which the specification might be implemented thin NASA. 

- o EM-16 MIL- STD-810E, &14IL-STD-883A, Critical Revievp & Gommentar^r 

was written, and issued September 30/1975- 
This was the outcome of the last of the specification analysi s tasks, re- 
quired by the study contract, and advanced copies of the completed EM; were 
made available to NASA early in October 1975- 


.3-3- Spscificatiozas Task,. Assiimptions r Sex'-eral Assumptions: Were made to ' 
facilitate the general specifications analysis -These are listed as follows: 

3 .3.1. Issue Da,tes ; The particular specification under analysis is the W.atest 
current issue in all cases- Ho’wever, where previous issues form a part of thn 
specification/standard as. in the case of 1III-STD-883A, all such praxdou,s. issues 
were included in the analysis. ; - 

3 .3*2, Sub-Tiers : do the e.;<tent possible/partinent, all sub-tiers of a major 
specification were izicluded in the analysis task, and. the interaotions were • b 
shown. 


LOCKHEED MISSiLES & SPACE COMPANY. INC. 


3»3»3« Cost Aspects of Jlnalysis : In cases where cost aspects of spBcifications 
were not readily identifiable from programma-tic data^ it was assumed that 
Li'-iSC estimates would be acceptable j provided suitable rationale was included 
to re -inforce the validity of such estimates. 

3.3.U. Cost •Quotations. & Dollar Savings ; ■^•^fherever possible, cost information 
associated with the specifications analysis task is' expressed in "Percentage 
of Total Program Costs (Percent TPC)'J bTiere dollar figures have been quoted; 
i.e, cost par page of documentation, the dollar val\xes represent those in use 
a.t LMSC . • , 

3 . 3 *- 5 . Terminology: As used in the analyis of : all specifications examined, 

the term SPSCIPIC AT ION denotes any document containing requirements, OTHPR. 

THAN TECHNICAL PERFORMNCE PAP.At'IETERS , routinely imposed on programs as a 
MSA practice. 

3 .I 1 . MIL-D-1000, Engineering Drawings & Lists; Analyses : firing the course 
of the MSFC stiidy, a. cursory examination was made of the cost impact on program 
of the 'three forms, and 10 categories of drawings imposed by MIL-D-1000. At. 
the outset of the cirrrent study NASA/HQ expressed interest in the MIL-D-IOOO . 
hierarchy of specifications, and the large sub-tier of documents comprising the 
hierarchy. An investigation was made of 'bhe specification, and the first of 
a series of SMs was issued on the subject, EM-2 issued January 7}^-97^j further 
detailed the preliminarj'" work begun in the MSFC study, and charted the complete 
documentation tree showing the total dependency documentation list associated 
with MIL-D-1000. Also shown was the fact that if Form 1, the most costly and 
complex of the three Forms ' cited by the specification, is invoked; 38 sub-tier 
documents are also invoked. Similarly, in cases where Form; 2 is specified, 12 
sub-tier documents are involved also. Finally, should. Form 3 be specified, only 
li sub-tier documents are aut opiatic ally invoked. All of these data were a.pplj.ed, 
and based on several programs IMSC has conducted, a cost ; comparison chiart was 
made showing both drawing costs for the; three Forms, as well as QA. costs, assoc- 
iated with these Forms. The complete EM appears in the A.ppendix, but for ea.se 
of reference, Figure 3 -1 is reproduced on Page. 3-h of this report. 

3 .li.1. MIL-L-iOGO Re-'ijrite : An outcopie of the ;NA-SA Low Cost Spectficatidns 
committee deliberations, was that LMSG should furnish a Low Cost version of 
the MIL-D-IGGG docuraent, incorporating all sub-tier documentation believed :. 
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pertinent to such a Low Cost version. This task request was conveyed to LMSC 
by the NASA/HQ COR at the Jan 21,1975 meeting, at IT-ISC, and the task was 
undertaken. At the May 25 meeting copies of the Low Cost Version were handed 
to the CORj and as has been stated, the version was not written as an EM at first 
The intent of the version, and the format adopted for it dictated this decis- 
sion for the following reasons: 

(a) NASA, could use it direct Ij/" for comments, and if fa,vorable react- 
ions were received impose it directlj^ on programs. 

(b) The f orm.at included rationale for the changes proposed in the 
restructured document, arranged in such a manner, that these 
rationale statements could be excised should NASA wish to impose 
the new version immediately. 

Although the re-written specification was not given an EM no., initially^ it 
appears in the A.ppendix, as EM L0PP-’20, De.c 10/?5„only to maintain continuity. 
Essentially, the new version shows how fiurther cost savings beyond those real- 
izable by imposition of Form 3 of the original document, can be effected. 

A.n example of such a savings is the use of sketched dra.wings,: where the lines 
are not ruled> but all dimensions are furnished, sufficient to permit hardware 
to be made from the sketch. 

3.1!.:. 1^1. MIL-D~1000 Analysis : On August 26 a short EM, EM LCPP-13 was prepared 
to furnish to NA.SA more details of the m.ethodolog;/' used to perform the MIL-D- 
1000 re-write task, and the ana.lytical details which emergea during the. task.: 
This sub-task was required by ths: Program S,Q.W. 

3.U.I.2. MIL-B-lOQQ Implementation Commentary : Also required by the S.O.W. 
was a short commentary to facilitate: implementation of the. Low Cost version of 
MIL-D-IOOO . . This was prepa.red, and tra-nsmitted to : MSAi during the 3.ast week in 
August 75* This EM,: LG PP-lii is included in the .specifications section of the 
Appendix.' 

3.5- Ajialj/'sis of 10 SeJ.ected Cost Driving Specifications : E?4 LCPP-3. included 
in the Appendix,'; treated the subject of Cost Benefit s Derin/able from Chai;igeS 
to 10 Selected Cost Ik’iucLng .Specifications . This task, arose' from a task imder- 
taksn diiring the course of the MSEC study, which seleGt.ed'.,20 specifica.tions, 
and explored the degree to which these specifications drove program costs . 
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The preird.se was made for this analysis of the current study^ that 10 of 
tlie 20 selected cost driving specifications could be changed to be more cost 
effective. If changes were made, to what degree could cost savings be realized 
was the question that the analysis sought to answer-. The analysis used two 
NASA programs of known lovJ cost as the basis for estimation: 

(1) MM-73 (Although this was a JPL program, for purposes of the 

analysis, it was considered to be similar to any other 
program managed by a field, center) 

(2) A.tmosphere Explorer (AE) G-FSC and RCA Corp. 

The format of the study analysis was to sjnopsiae each of the 10 cost driving 
specifications briefly, to pez-’mit visibility into programmatic areas impacted 
by each driving specification. The next step was to estimate from programmatic 
cost data, the costs attributable to ea.ch specification. Low Cost changes were 
then prescribed for each of the specifications u.nder stndy, an.d finally, the 
cost savings potezitial of each of these change's was estimated. Outcomes of thi 
estimation, process vs the- sizggested changes were compiled into a simple matrix 
To permit visibilit^r into the outcomes, this matrix is reproduced for reader 
convenience, and appears as Fig . 3 » 2 .. on Page 3 -7 • 

3.6. Specification Commonality Anal7>^sis for 6 Selected NASA. Programs : This i ' 

analysis sub-task of the overall specifications analysis effort, wa.s com- 
pleted in January of 1975- Ehl lGPP-U Mas written to describe the outcomes, 
and the, -SH was provided to N.A.SA Jan I6 197^* The programs selected to be. stud- 
ied in terms of specification commonality were : 

(1) Nimbus G.OSFC/General Electric Co : 

(2) Atmosphere Explorer. GSFC/RCA Gorp, 

(3) HEAQ A/G, -MSFCAJd^ Corn. . . 

: (It)_ Pioneer-Venus. iVRC/Hughes Aircraft Corp. 

(5) M‘7M-73- ' JPl/Boeing Corp. V- 1 

(6) :P 71-2 USAF/SAlfSO/STP. LM8C. 

The original intention was to select 6 programs, but availabilit3'' of program 
data was such that a JPL program., and the ,L!iSC P. 71-2, were included to maTce : 
up the 6, 'in the absence of NASA data fox’' 6 MSA. prog.rams. . 

3 .6»1. Outcomes : .It. became obvious earljr in the: analysis that very little ; com 
monality existed among specifications, program.-to -program, each program tendin 
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COST -DRIVING 
SPECIFICATION 

invoked/applied 


NHB 5300 . U(1a) Rel. 


NHB 5300.ii(lB) Qual 


NHB 5300.li(lC) Insp 


HB B300:jinA)Solde 


NHB 6000. 1(1A) Pk 





Test 

MIL-STD-8O3 Methods 

7.9 

Config'n 

NPG, 500.1 Mgmt. 1 

0.35 

Envir'l 

MTL-STD-810B 

3.2 

Interch' ly 
MIL-I-8500' 

0.2 

Eng Dwgs 

MlL-D-1000 

6.85 



0.2 


6.60 


ALL VALUES LISTED ARE IN PERCENT TPC- 


MAJOR COST REDUCING CHANGE APPLIED (ASSUMED) 


Eliminate Parts Population Prediction. Use dynami( 
simulation. Eliminate Reliability Demonstration. 

Delete Formal MRB Rqmt. Substitute informal MRB. 
Simplify Mgmt System. Re.duce Reporting. Delete 
O.C. Curves. Simplify Inspection Instructions^ 


Inspect at Highest Possible Level. Use Double 
Sampling. Minimize Insp'n points in process. 


Eliminate NASA certif 'n for contractor solderers 
& inspectors. Approve process/training only. 
Uodate soec to include latest illustrations. 


Eliminate this General Specification. Detail 
Rqmts in S.0.¥. ner mandatorv minimum. 


Limit Life Testing. Limit Rqmts for ARay Tests. 
Limit Test Repetitions. Specify Mimima not Max' a. 


Eliminate Batch Traceability. Elim. Mat'l Certs. 
Simplify numbering & Codes. Apply only for >3 S/C 


Eliminate Thermal Vac at syst level. Apply at 
box Ivl. Minimize Test Repetitions. Limit duration 


limit to Programs with several S/C. Std'ze 
parts & components. Use Std Designs as possible. 


Reduce Form call out frm Form 2 to Form 3. 
Limit No. of Copies. Simplify QA & Insp Rqmts 


A P ercent Savings due to Changes 
NeWj Reduced Percentage of TPC 


Total Program Costs 


3 .2. Affect on Program Costs of Soecification Changes 


27.50 


27.75 














































to "Go its ovin way" with respect to the specifications imposed upon the 
program contractor. Co.st implications were difficult to determine with any 
accuracy, with one exception. In the area of Product Assurance, safficient 
general cost data were available to permit the estimation of assurance costs 
as a percentage of the total program costs. Therefore, a simple commonality 
Matrix for the specifications was constructed, and this appears as a three 
page tabulation in the EM, and together with this, a tabulation of assurance 
costs was provided. To this cost tabulation of assurance estimates were added 
verbal estimates from MSPO, and ARC, concerning their on-going programs, and 
since these do not have the advantage of "All costs are in" credibility, 
costs for OSO-1, aiid ATS-B'’ programs were added. This cost tabulation appears 
below: 


Program 

Contractor/Center 

Ass\irance Costs ^TPC. 

MVM-73 

Boeing/ JPL 

6.1 

AE 

rca/gsfc 

3.1 

Pio. Venus 

a-iag/arc 

5.5 (est. prog current) 

Nimbus G 

gfo/gsfc 

6.2 

HEAD 

trw/msfc 

6.0 (est. prog current) 

P71-2 

LMSC/USAF 

5 . 1 . 

OSO-I 

hac/gsfc 

5.6 

ATS-F 

Fairchild/GSFC 
LMSC Flex-Rib 
antenna. 

8.1} 


Some reccomendations were made in the EM, two of which gave rise to: othar 
tasks within the . oyer all, spscifica.tion analysis portion of Stud;/ , Task 2 . ,, 
these 'two‘ are ; . 

(1) Revise the .NHB 5300.1} .( IB) & (1C )' Quality Assui’ance ^ Inspection 
Sp a cifi cations.. 

(2) - Plan, aiid Instituts, a: MASA Gentrai opBciflcations Control Office. 
The first r e common dati an formed the subject of S! IOPP-5, and the second form- 
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ed the basis for Ei^ 
the May 5th me e ting j 


LCP?--5-(Per information received from the NASa/HQ COR at 
I'lASA is proceeding with the institution of a Central 


Specification Control Office.) 


3-7 A Plan for A NASA Central Specifications Control Office (CSCO): 

3 . 7 . 1 . Analysis Over'^n.ew ; Ai'ising from the specifications Commonality analysis, 
this plan was undertaken to indicate to MASA/kQ the need for such an office, 
and a method by which such inight be instituted. The M written to describe 
this planning was in two parts. The first part depicted the current state of 
affairs in both NASA and the DoD, and showed the relationship of NASA to 
the NASA and Tri-Services Review Board, highlighting the fact that NASA 
gains very little advantage from participation in the Board. The specification 
release, and modification system was graphically illustrated. In part two, 
functions of the CSCO were outlined by an illustration of a functional flow 
diagram. In addition to the function flow diagram, a graphical representation 
of the aims and objectives of the CSCO was presented, together with the system- 
ic process of the office, and a detailed mu3-ti-path work flov? diagram. The 
systemic logic underlying the functions and work-processes, was described, 
and all of these planning data were incorporated in M LGPP-5 issued April 
10 / 75 . Much of the planning data included in the M is graphical in nature, 
and for sal:e of bre^/lty is not included here, as the M is part of the A.ppendix. 
HovJever, a diagram of the Systemic process is shown as K igure 3.3 . Page 3~10, 
to facilitate understanding of the operation of the proposed CSCO. 

3.8 Critical Analysis & Commentary, NPIB 53QO.i4(l3) ; As originally ercri.sioned, 
the objective of this task was to re-write NH3 5300.[i(l3), combining within the 
re-written version, the main elements of NHB 5300.U(1C) so that a composite 
document embodying both Quality Engineering, and Inspection Services would res- 
ult. Discussion with the NASA./HQ COR in January of 1975 amended this task to 
combine the main elements of both the existing docijments, but instead of re- 
writing them, a detailed commentary with recommendations for changes was to be 
the task output. The documents are arranged by chapters, and within the chapters 
paragraphs are ordered in numerical sequence . Each paragraph has a letter des- 
ignator praceeding the numeral; vis;-; Chapter : 5 , 1350P Procurement Dociirient s ; 

The approach to the task was to analyse the contents of the .docimients; paragraph 
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-by-paragraph, giving the current statement of the paragraphs in an "Is" 
column^ and a "To Be" column in which the recommended changes, and the ration- 
ale supporting them are given. This approach resulted in a tabulation of lU pgs 
in length, an excerpt of which is furnished belovj: 


‘Chapter ' Is How ' 


Chapter "To Be" 


1 B 701 Inspection & Test 


Planning : Sets forth the aim 
and intents of the 'inspect- 
ion and Testing Operations 
to be provided by the contra -1 
ctor, & his supplisrs. 


13702. Test Specifications: 


Covers the provision, and 
prapax-ation of test specs. 
States what specs are to incl 
uds'.'- 


13703, Inspection and Tes't 


Procedures , 


Covers the provision of 
■suitable procedures f'Or 
regulation of these operation 
and specific instruction of 
operating personnel assigna 
live a 13 point outline of 
content reouired. 


IB7OI : Again, the statements are general in 
nature . They state what is required of the 
contractor, not what is required as specific 
activity by his QA pers-onnel/organization. 

This clause has three actual functions involved 

( 1 ) Inspection Services. Provides Inspection, 

( 2 ) Test Operations. Plans, Executes Tests 

(3) Design Engineering: Design sets test limits 
The QA function is again one of surveillance to 
determine compliance only, NOT as has been the 
case, effectors. 

Recommend : Delete Paragraph/ Re-write to state 
■Quality finite functions. . ■ ' - 

, 13702 . Covers the entire gamut of rqmts. 

Include s Etrv ir onment s , safety, re liability, 

test parameters & tolerances, .adjustments, 
rework, repair, mainueiiarLce, ■’ data recora-'’ 

in-g and analysis, re-test, and test results 

reporting. P.ole of -Qualit;/ is only to assure 

that specs generated include such variables. 

Recommend: Re-word to show specific quality 


responsibility only. 

13703 •The inference can be, and in the past 
has been made, that a contractor's quality 
organization is to carry out the, activity 
described as necessarj/: in , pro^'/ision of these 
Is) ■orocedures. Of 13 points covered, only 
item 13 Hon-conformances is of direct c-oncern 

to QA., The others concern Test Ops planning,' 

Engineering, .Safety Engineering, and Inspect- 
ion Ser,\does. The -QA function, is^^ t 
procedures are; available and contents are 
as prescribe d., - 

Recpmme hd : Deletion, and relocation of rqmts 
in test, ispection, & safety documents: or 
re-write to show; Specific QA- responsibility. 
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The second part of the analysis was an attempt to show costs associated 
with Quality Assurance to programs where either N^IB 5300*U(13), (IC), or 
the DoD Quality Specification MIL-Q-98^8 a was full;/- imp lamented during the 
program^ and some of the economies possible. 

A Quality Costs Tabulation was made of some cost sta.tistiGS on typical space 
programs in the 90-100 million dollar range , as follows : 


Quality Program.Sj General Statistics 

1. For typical space programs in the 90-100 million dollar range, Quality , 
Costs range from 7-11^ total program costs (TPC).* 

2. Quality Costs for t^.'pical programs range from 3h~h^% of Manufacturing Costs 


3. Typical Space Program Quality Engineering Costs range from 23~37m ot 
Total Quality Program Cost s . 

Typical Space Program Inspection Services, Costs range from 63~71% of 
Quality Progrem Costs . 

Typical Ra.tio, Quality Engineering (QS) to Inspection . , . , . . . 1 13 approx . 


U . Economie s Possible ; Atmosphere Explorer ( GSFC/RCA ) Quality. Costs 3»8;& TPC 
Attributed to; Integra.tion of Quality/lnspection, Reliability, and . 


C on f ig 'or at i o n Management , 
program activity. 


effort S: olanning into one oyer ail 


Finally, Qua.3.ity Engineering, and Inspection tasks performed on a typical , . 
program, were broken out by task, and referenced by specification paragraph, 
giving the Percentage of Quality Program Costs for both sets of detailed 
-tasks, and the full Quality Snginegring Program , and Inspection Program 
Costs as Perce ntages of Total Program Costs. For ease of reference, this / 
table is reproduced herein as Figure 3'U.j Page 3-13 . (Ihe complete . EM .for ' 
this task, wMch contains the. data. raproduced partial.ly herein, is incliided . 
in the Appendix) 
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13206 

IB^Ol 

L3300/^( 

1B205 

1B206 

13300 

1B600 

13703 

1B706 

13703 

IBS 01 

13905 

13202 

IB805 


13701 

1B202 


Major Quality Tasks.: Typical Q uality W3S (internal) ^QPC 

Quality Hanagsraent 1.55 

Vendor/Supplier Surveys 0.55 

)O0uality Docinientation (Preparation 6 ; Updates) ll.OO 

Quality Audits 0.35 

Quality Planning (Inspec'n S: Test, Work-flovjs & Plans) , 1,20 
Quality Liaison (Engrg & Sngrg Support Serirf-ces) 2.ii0 

Quality Liaison (fifrgj & Mfrg Support Services) 7.10 

Quality Liaison (Test Operations^ d Test Support Svcs) 2.20 
Quality Data Retrieval^ Reduction^ 6 : Reporting 3*50 

Inspection Services Liaison 3*20 

MR3 6 : Corrective action Follow-up. (average only) 1.50 

Measurements Stds Lab ^ Liaison & Control . 0.25 

Quality Training Certification 0.70 

Customer (MSA) Liaison 1.50 

Major Inspection Svcs Tasks: Typical Qt/BS (internal) ^QPG 


Inspection Services Management , 
Inspectors Training & Certification 


IB 70 I 1 Quality Engineering Liaison 

1B5o 6 Receiving- Inspection 

1E600 In-Process Inspection 

IB 705 End-Item/Final Inspection 

I370ij Test Sitrveillance 

1B801 Non Conformances Reporting 

IBllOl Pre-Shipment & DD-250 Inspection Activity 

13905 Calibration Inspection (Measurements i Stds Lab Liaise 


2.25 

2.50 
3.20 

12.50 
26. UO 

5.15 

12.50 

3.50 
, 1.25 

l)0'.50: 



Fig. 3-U Quality Engineering & Inspection 
Costs of Major Tasks 

The -M, LQPP -6 was f uini shed, to IOBA/KQ in early. March 1975, and at the May 
75 meeting at, DISC, the COR presented a written list of comments on the 
Critical Analysis. These were prepa.red by the NASA/HQ Director of Reliahilaty 
<£c Quality Assurance^ As might he e:<pected, the reaction to the Corrmientai’y was 
mixed, with both commendatory coraments, ' and oriticism included in the review. 
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Essentially, the review was favorable, applauding the idea of amalgamation 
of both the Quality, and Inspection documents into one volume, but criticizing 
the deletion of some of the well-established quality functions such as 
reduction of the number of inspections per program. At the request of the GOR, 
a detailed response was prepared to the NASA/rQ comments, and at the mid-teimi 
re-'.d.ew in June, the entire, commentary, NASA review, and LMSC reply, were 
discussed with the HQ Director of Reliability and Quality Assurance, A. consensus 
of opinion was . reached, to the effect tliat. the 'LMSC commentary would furnish 
a major point of departure for the re-write of the Quality and Inspection 
documents which NASA, expects to undertake internally. .. 

3.9 Criticai Analysis and Commentary, MIL-STD-8IOB, & MIL- 5 TD- 883 A ; .This 
analyis was the final specification analysis of the study, and perhaps the most 
difficult to perform. 

3 . 9 .1 Objective ; The objective of this task was to highlight the anomalies 
and overlaps erxisting between the two documents; show that much of the contents 
of MIL-STD-8IOB, Environmental Test- Methods are" not applicable to spacsGraft; 
and indic 3 .te that the MIL-STD-883A document prescribes to a large degree, the , 
Test Methods that apply to microelectronic .devices, thereby making unnecessary 
further testing to comply with MIL-STD-8IOB. , 

3 . 9 . 2 .. Analysis & Commentary Approach ; For both of the documents the approach 
taken was to review each of them by section and paragraph. Each paragraph was 
s^mopsized in the written on the entire : analysis ( EM .LCPP-I6 issued Sept 
30 197.^ ), and to each paragraph synopis was appended a. Comment statement. 

The comments briefly considered the pertinence of the paragraph imder review,. . 
suggested changes. to improve cost effectiveness, where such changes appeared 
feasible, recomxaended deletion in instances where this appeared advisable, 
and explored, the overall costs associated with the two documents as applicable 
to two prograrasj kn/kI -73 and. AE. These two programs .were selected since 'they: . 
were among the earliest to make wide use of microcircuitry, and microelectronic 
devices, and were programs undertsl'en during the period when, the micrO-devicss 
state-of-the-art was still maturing. . . 

3 .9.3* Test Cost General Summary ; ' One.of , the cost , drivers . to both the programs 
considered was Space Simulation Testing,l.as prescribed, in MiL-.STD- 8 lQB . A. 
general, test cost-summary for; these two programs was synthesised from the cost 
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data available, and for convenience this s-ummary is reproduced below; 

Total Test Costs ^TPC Ttl. Env Test Costs Ttl Space Sim Costs 
MWI-73 18 . 0 % TPC 3.8f4 TPC “ ~ " 1 . 7 ?^ TPC 

AE 16 . 2 ,^ TPC li.O^ TPC 1 . 9 ^ TPC. 

-The inference was made from the co,sts above that had- not space simulation 
testing been conducted, a savings to both programs of .1,7 & 1,953 T,FC might have 
been realized. For programs of roughly 100 million dollars, ^2 million saving 
is an appreciable figure. At present considerable controversy surrounds the 
subject of space simulation testing, and opinon appears polarized, as to 
the. efficacy of such testing. LMSG is of. the opinion that if such testing is 
required at 'all, it should be on a limited scale, and at the component level 
only. Testing of microelectronic devices is a matter of multiple replications, 
and while a cursory examination of the costs attributable to MIL-STD-883A 
required testing was made, the data are of considerable dubiety. Eli LGPP-3 
(which see) inferred that su'h test costs could amount to 7.^5^ of TPC, but 
this valtie is difficult to consider as a nornativs figure, as high mortality-' 
rate of microelectronics, and associated high costs, undoubteldly were distort- 
ed due to maturation of the state-of-the-art by which the deuces are produced. 
It was further inferred in EW-3 that 1.5--2.5^ TPC might have been Served by 
elimination of one or two steps of test replication. NASA found it necessarj'' 
to establish line certifiction methods to improve the yield of the micro- 
devices, and in the early days of production such certification was necessary. 
Curre.ntly, some of the certification testing is no longer, needed, as major 
improvements in the state-of-the art have been effected. The savings figure 
would appear to be reasonable therefore, or even conservative. 

3 .9.1i. Outcomes : The major ^ outcome of the analysis was a series of specific 

re commendations,,, which for, reader convenience, are listed below ; 

o Review and combine the two standards, into one conjoint MSA 
standard, to cover all aspects of environmental testing at all 
hardware levels. 

o Specify in the RFP,. or other contractua.1 document, exactly, the type; 
and. level of testing the Government requires to be performed.: Make 
the contractors ’ test plans the binding docum.snts, for type, level, . , 
replication, and schedule: for all testing performed., 
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o standardize device testing to the maximum extent possible, 
with respect to methods, replications, sample sizes, para- 
meters, et al. The current test methods and standards tend to 
be too general. 

o Minimize test replications to the maximum extent possible, 
and in all cases where a test at the next higher assembly level 
will suffice to demonstrate merit, use this technique to curtail 
the test load. 

o Minimize/standardize the amount of test data reporting, and 
test documentation required at a.11 program levels. 

• o Assure that Quality Assurance and Reliability requirements ; 
specified are adequate for program assurance/validation, rather 
than the maximiim possible, as is the current practice, 
o Standardize the number of devices available for a program, 
and do not design, new ones merely for the sake of doing so. 
o Until such time as the present MIL-STDs controlling testing 


are re-written, and re-issued, ensure that these documents imp- 
osed to control the overall program test process are the latest 
issues available, and thmt a list of tests not required (Test 
Scception List) is appended to each purchase order j particularly 
at the basic de'xLce level. ■ 
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SECTION h 


PROGRAM EFFECTS RELATIONSHIPS (PER) SURVEY 

I4..I. Task Objectives : At the close of the MSFC study, LMSC had completed 
and delivered to NASA a Per Handbook (prelinu.nary) . PERs are line graphs, 
ban charts, data tables, and families of curves which identil^'", and portray 
the key parameters of alternative practices, both management and technical: 
within a general program pra-ctice area. These parameters and their relationship 
to program costs were expected to serve as an estimation tool for senior 
program planners, and decision making personnel, so that , relative costs of 
historical practices could be contrasted with alternative low cost practices, 
and estimates of potential sai/ings could be made. The objective of Study Task 
Ii was to determine b2/" means of a NA.SA/industry survey, the potential acceptance 
of such a too 3 ., the utility of a PER Handbook, and to obtain suggestions for 
new PERs of value to the expected users. 

h* 2 . Task Approach : A two-step approach was used to attain these objectives: 

(1) Prepare a comprehensive PER Questionnaire , for either hand 
delivery, or mailing to participants in the PER Sirrve;/, 

(2) Personal Interviews with, and/or seminars for, the SuTTre^/" 
participants to obtain first hand information concei-ning the PER.S. 

Interviews planned and accomplished were as follows: 

0 Ten (10) NASA Field Center Project/spacecraft managers 
0 Ten (10) NASA Field Center fuiictional Division Managers, 
o Three ( 3 ) NASA/HQ Program Managers (OSS, QA., OMSF) 

0 Nine . (9) Industry.'' Program/Functional Division Managers represent- 
ing at least 5 companies. 

Interviews were conducted at; NASA facilities, and those of the responding 
industrial companies;, by personnel of the LCPP; Stud2/’ working, as a two man team.- 
An 18 page multiple choice quastionnaire was designed which included Practices 
A*reas,. Cost Impacts, and Priority Ratings for all programmatic areas of inter- 
est; and a foreword to the participants guaranteed them anonymity if they so 
desired. Instructions were, included in the foreword to assist in the completion 
of the document (LMSC-D 3875 i 4 b) .v with stress on the fact that re spon.de ntyS were 
encouraged to give free expression, to their views, b^r adding comments in spaces 
prc’/ided, in. addition to completing the multiple-choice block entries. 

h-1 

LOCKHEED MISSILES & SPACE COMPANY, INC. 


ii.3» Survey Re suit s ; Most of the sijrvey interviews were conducted during the 
months of November and December of 197^. vjith some few interviews in January 
and February of 197^* As was anticipated^ there was a.n appreciable lag in 
the time from interi/iew until the completed questionnaires were received, 
although many of the interviewees completed the questionnaires and returned 
them during the course of the interviews. NASA and Industry participants ware 
as listed below: and the number of completed questionnaires is included: 

NASA Industry 

Ames R.C. 9 Hughes Aircraft Corp 2-«- 

GSFD 6 J?L . 6 

JSC 9 IMSC 2 

MSFG 7 Martin Gorp (Denver) 3 

NASA/HQ 2 PhilcD WDL 3 

Total 29 Systems Devel. Corp, ' 1-!H:- 

TP.W Corp 3 

Total 20 

Selected tearas of personnel preparud, two (2) questionnaires. . 

Only one questionnaire requested from this source. 

Survey Final Outcomes : M LCPP-12 „ compiled and delivered to NASA .April 

21ji 1975^ contains not only the statistical analysis of the results, but also 
a complete questionnaire upon which appear the scox’es per each question. 

This M (included in the .Appendix in its entirety) when completed, led to a 
number of conclusions and recommendations being made to liAS.A/HQ, 

[j j-i.l. Conclusions .: Due to the length of the conclusions .seotion of the EM . 

only highlights are presented hereinj they are: 

0 0 nl 7 / l8^ of the survey participants believed that the historical data 
in their company/center was adeouate for the generation o.f meaning- 
ful PSPs (meaningful = . accurately ayiantified) , 

0 The PER Handbook idea is basically: sound, but data preclude its 
amplificaticn at present. Opinion on its use was polarized, with . 
many respondents averrixig that tihe handbook would not be used., ■ 
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o Risk considerations inherent in the use of Low Cost Practices/ 
and the lack of PERs capable of quantifying these risks was 
of considerable concern to NASA personnel, 
o Industry personnel were strong in the opinion that their compan 
les would do what is required of them by MSA, as stated in the 
program RFP, and PERs are of secondar^r importance to that aim. 
o Handbook format was believed to be satisfactory as LMSC had 
presented it, 

iiJ 4 . 2 . Recommendations : As an outcome of the statistical analysis of the surv 
ey results, .and the opinions e:>(pressed by the respondents, LMSC drew 8 major 
conclusions, the first three of which were discussed with NASA/HQ at the mid- 
term review in June 1975.- and had a direct bearing on the remainder of the st 
The conclusions appear below in synopsis form: 

1 , Discontinue development of PERs 

2 . Start immediately to develop a Work Breakdown Structure (WES), 
capable of accurate quantification of Low Cost Program Practices 
in future space programs. 

3 * Evaluate Risk Impacts of potential Low Cost Practices, also to be.gin 
immediately. 

h. Include Low Cost Practices in future program RFPs. 

9. Continuing quantification of Low Cost Practices to be undertaken by 
NASA personnel. 

6 . Folio vj up the PER Survey with an Aerospace Executive Level Survey 
on Low Cost Practices. 

7. If item 6 is implemented, prepare an Orientation Brochure and 

Survey Questionnaire, using results of this study and others as 
inputs. : I , 

8 . NASA should institute an Education/Promotion program ifh field ■ 
centers, to facilitate the use of low: Cost Practices.! 
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SECTION 5 

DATA ACQUISITION, (STUDTTASK l) 


5.1 General ; The Data Acquisition Task, Study Task 1, 'had as its objective 
the collection of programraatic data from NASA/HQ, NASA Centers, DoD, Indn.st3:y, 
and other sources, suitable for examination and excerption to facilitate the 
identification of Low Cost practices. In addition to those data which formed, 
the Data Bank for the MSFG stud 2 ,'", data we,re. sought vihich had particular pert- 
inence to the study of practices related to SSSiI, GSE, and cost collection b^r 
means of a V/BS. 

5 . 2 . Data Types Avadlable : Du.e to the interest in the P 71-2 Program (UiSC/USAP) 
expressed by NASA, as complete a data package as possible was collected , from: 
Program Office records of this program. A.s explained earlier in this report 
these data were instrumental in the preparation of EM- 7 which researched the 
program history of the progran performance, and reported: 

0 Program histox’y (technical) 
o Lists and descriptions of Program Components 
0 Cost data for program components 
o Programmatic cost data to permit analysis of (a) 

costs, (b) routine practices applied vs low cost practices. 

5 . 2 . 1 . M'VM- 73 /JPL/Boeing : , Contact was made early in the program with JPL 
Pasadena Calif, and this organization provided fairly complete data on the 
performance of the subcontractor Boeing Corp, Lhile data concerning the JPL 
cost breal'd own for the program were, considered essential; to the analysis of 
this Low Cost prografTi, .and its practices, LilSC was unable to obtain a cost 
breakdown which decribed the contract oerformance costs for the JPL in-house 


effort. A data file on this program, as complete; as NASA/HQ possessed, was 
obtained from the NASA/HQ Prog.ram Manager. 

5 •2.2. Other Data Source s : NASA./EQ furnished data concerning the following 


programs : 

o OSO-I o Pioneer-¥enus 

o ATS-F 0 Atmos.Sxpl* er . 

p Nimbus -Gr : b HEAO A,/C 

In addition to the above, NASA furnished assorted data 


oncerning general NASA 
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contracting practices, NASA Pops, PA Ds, and such other data as the COR thought 
pertinent to the study. 

5 . 2 . 3 . Data State : Despite the expenditure of considera.ble time and effort, 

LMSC was. unable to obtain a complete set of data for any one program. Such 
a data set would have consisted of the early NASA program decisions, the Phase 
A Program definition, early contractual surveys of possible contractors, the 
R?P, the contractors' responses to it, the contract data concerning seleGtion& 
award, early funding agreements, the contra.ct document itself, subsequent chan- 
ges to the contract scope and intent, programmatic records of performance, 
contract finalization, fee awards, and detailed cost brealcd own by contract 
period. Data: provided by KA.SA were fragmentary, and data of the type desired, 
and solicited by letter from the severad. contractors involved in the NASA 
programs listed, were either not available; or again, fragmentary. During the 
data acquistion phase effort was expended to obtain data whose subject was 
programmatic costs of SS?r.I Activity, and GSS used for programs. These efforts 
proved only parti all;/- successful. . - 

5.2.ii. Data Problems : In atialysing the data that were available,, it became 

evTident that specific data concerning GSE were almost unavailable in usable 
form. tBS reporting tended to list GSS as a single block against which costs \ . 
xor this item ware collected. Wo GSS lists of any value Were kept as a program 
record, nor were valid cost data available by specific GSS item. GPS was seldom 
identified as separate from GSE, nor, with the exception of MW- 73, was GSE 
inheritanca, program-to-program readily traceable . In an attempt to obtain more 
GSS data, a GSE questionnaire was. compiled. This questionnaire, which will be 
treated later in. Section 7 of this peport was both ma.iled to participants, 
and also hand delivered during the course of the PER Sui-’vey. The responses 
to that siurvey will be: discussed in Section 7, but essentially, data received . 
were insufficient to allow the GSS task/to be continued as originally planned. 
Similar difficulties accompand-sd the data acquistion ta.sk for the: SE&I portion 
of the st‘ady, as discussed in Section 6 of this report, such that the scope 
of this portion of -bhe study act i-^dty could not be as extensile as -was planned. 
5 . 2.5 Data Usage : An integral part of Study Task 1, was the analysis perfor- 

med on the P 71-2 (.SESP) data. A.t IfASA request, the following activity was 
completed using the_ SS3P data; 
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-j:- (a) A comprehensive listing was made of P 'J1^2 SESP components 

(b) A listing was made of Component Sources^ Source Points of 
Contact^ and component spares, 

(c) A listing of component design dravjings^ together with drawing 
copies was made, and where possible, the listing was cross- 
referenced with a listing of component costs. 

(d) A detailed Cost Breakdown of the SESP program costs was made 

and included in this listing of costs were data concerning program 
perform.ance, program history as affecting costs, and program 
schedules. All of these data were combined into E.M LGPP-10,1 May/7^ . 

(e) ' E?'I-7 was prepared which gave a, complete as possible program hist- 

ory of P 71-2 (SSSP), from both the technical •'.d.ewpoint, the, com- 
ponentiy listing used, and the overall design disclosure. / 

,(f) Eti-U was prepared, May l‘3/7^, which furnished a sjmthesised 

ar.alysis of the cost sa^^/ing which might have been effected by the 
application of Low Cost Practices to the SESP program 


* These items were furnished at NASA/KQ request to Aerospa.ce Corp, for inclus- 
ion in the NASA Components Data Book, whn.ch that company was under NASA/KQ ; , 
contract to prepare and publish under the title G . A , S . H . ( Catalog of A.vailable 
Standard Hardware) 

^.2,^.1. Uses of Other Data : The other p.rogrammatic data listed herein were used 
to the extent possible to perform the remainder of the analyses undertaken in 
supoort of Study Task: 2. hllL-STDs, Handbooks, Specifications, RA.SA Handbooks, 
STDs, Specifications, PoD Directives, and Industry Standards were used as requ- 
ired, and these were furnished from the LHSC data bank.: 

^.2.6. Conclusions : Apart from the scarcity/unavailability of programmatic data 
which impedsd the progress of some tasks of this study; a conclusion emerged 
clearly ■from, the Data Acquisition t?vSk. Detailed aost data of the t^rpe essential 
to permit visibility into historical program pra:ctices, ,do not exist to any 
marked degree , The data available were not acquired , and analysed in a manner : 
such that visipility into program practicas was possible., as ^ until the recent, 
NASA emphasis on low costs, aerospace companies reporting data, have neither 
been contractuslly required, nor made it a practice to descend to minute levels 
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of cost data recording/reporting. 

$.2.6.1 Outcomes Af fe cting the Study : The lack of complete data, and the 

state of the data available led to a re-direction of the LCPP study. This 
re-direction took the following form; 

1. The task of generating PERs 'Was curtailed due to lack of 
suitable data. 

2. The SE&I analysis v;as completed to the best degz-ee possible, 
but was curtailed due to lack of suitable detailed data, 

3. The GSE analysis was completed similar Ir, but the task was 
also curtailed due to. lack of suitable detailed data. 

* h. The specifications analysis task continued for additional spa ca, 

$. A qualitative review was made of the impact of low cost 

practices upon program risk, and a sxurvey of the state -of-the 
art-techniques for quantitative deteiTnination of program risk 
was prepared. Outputs of this task were EMs LC.PP-1$ & LCPP-19 
and the details of this task are discussed in Section 9 foD.lowing. 

6. The need was envisioned for a comprehensive VIES, so constructed 
as to ser'/e as : 

(a) A Standard tJBS for NASA, unmanned space programs" 

(b) permit acquisition of detailed prograia costs by 
hardware and labor function costs. 

(c) Demonstra.te the cost impact of applying low cost practices 
By direction of NASA: work was to begin on this task immediately, 
and a desci-iption of this task forms Section 8 of this report. : 
The EJ-'I genera.ted deocribing progress of this task is LCPP-17 . 

Items listed above, and identified with an astex’isk form the re-directed 
tasks of the study as pz"’oposed by LMSC, and approved by NASA/KQ June 197$ • 
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SECTION 6 


SYSTEMS ENGINEERING & INTEGRATION (SE&l) IMPACT ANALYSIS 

6.1. Analysis Objectives : There were three (3) objectives associated with 

this analysis : 

(1) Determine the Costs of SE & I activity 

(2) Identify the functions and practices associated with, and 
contributing to these costs. 

(3) Identify and recommend alternative practices, which when 
implemented, will reduced SE 1 I costs to fut'ore space programs. 

6.2. Analysis Approach ; The approach to the analysis was as follows: 

o List and describe the typical SE & I functions performed 
o From historical programmatic data, supplemented by estimates 
as necessary, determine the costs involved for each function, 
summarizing the totals by SE&I function and program. 

0 Determine the percent of TPC for each SE&I function, the 

percent of each program, and the average for all programs consid- 
ered. 

o Propose alternate SS&I approaches/practices believed to reduce 
costs, or eliminate some cost items, 
o Estimate the cost savings expected to result from implementation 
of the recomraended Low Cost Practices . 

6.2„1 Analy s is Ass umpti on s : Modified practices suggested for application to the 
historical programs used as a data base, could be applied with equal ease to 
current similar programs, was. the first assumption made. The second assumption 
was that the practices suggested could be implemented on Shuttle era payloads, 
using modular spacecraft, and re fur bishable/re -usable standard components, 
and/or modules. 

6.3. Analysis Problems : Only very limited data at program top levels were avail- 
able, and these were from the programs listed i n Fi? 6.1 Page 6.2. Another 
difficu].ty was that the top level data available from the programs listed, con- 
tained very few breakdowns by function and associated cost. Data for the paper- 
work, integration activity were almost non-existent, and the only integration 
costs available concerned hardware integration rather than paperwork. Since 
hardware integrations costs were not to form a part of this analysis, separation 
of the hardware :and software costs proved impossible, and with the proscript- 
ion against hardware costs also, the integration task was deleted from the 
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Fig. 6-1 SE&I Data SourGes/Availability 


















































analysis j by necessity. The final major problem with the analysis was 
that for the data base used^ there were mariced inconsistencies in practices 
in the areas of Program Pefinition^ RFP and WBS preparation, and program record- 
keeping . 

d.i; Analysis Outcomes : Early in the analysis it became apparent that the 

term "Systems Engineering" is very broad in meaning, and what is actually 
landertaken under this category by aerospace contractors varies very widely 
from one contractor to another. Similarly, costs acquired and reported under 
that catgory are subject to considerable variation, program-to-program; even 
though the programs are quite similar in nature. Fig. 6.2 appearing on Page 
6.I4. illustrates such variation, and suggests that where high inheritance of 
technology, and equipment from a precee ding program or programs , exists , it has a 
material effect on systems engineering costs. 

System Engineering even varies in formal definition by the Government specific- 
ations imposed to define it. There are similarities and overlaps present in 
the two major system engineering documents , MIL-STD-i 99 and MIL-STD-88I 
which are imposed on programs, but there are also dissimilarities and differences 
The inconsistencies examined , and recorded in some detail in the Efi produced 
covering the subject, carry over into the production of the l^TBSs. produced for 
the programs examined during the course of the analysis je.g. at one extreme 
is the HEAO program with a highly detailed inclusive of integration 
activities, and at the other is the ,AE program which is far _ simpler as a VJBS, - 
and included the three basic systems engineering functions only. 

6 ,[|.l Cost Outcome : In an attempt to demonstrate the disparity, program- to-prog- 
ram of systems engineering charges versus program costs, a graphical set of 
relationships was produced, and for reader convenience, this appears on Page 
6,9. as Fig 6.3. This figure also sets forth the major conclusions arising 
from this graphical exercise. 

6.5.' Conclusions ; EM LCFP -8 (issued April 21 /?^) had a number of minor conclus- 
ions, and two major ones appended . The two majoi' conclusions are reproduced, be- 
low, as follows: . . : . ' 

0 It would appear that MSA has been paying from 2.8 - 11 . TPC 
(for each of the programs reviewed) for something called Systems 
Engineering, sometimes called Systems Engineering ^Integration. 

■ - 6-3v. ■ 
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Spacecraft 

Program 

Cognizant 

Agency 

S/C Contract 

System 

Engr. 

Cost 

Total 

Program 

Cost 

Avg. 

Year 

Expend. 

S. E. % 
of 

T. P. C." 

Type"* 

Program 


OSO-1 

GSFC 

Hughes 

$2. 772M 

•$34. 281M 

71/72 

8. 1% 

F/0 


ATS-F 

GSFC 

Fairchild 

4.3 (E) 

75.4(E) 

71/72 

5. 7% 

F/O 


SMS 

GSFC 

Philco 

3.117 

42.027 

71/72 

7.49% 

New 


AE . 

GSFC 

RCA 

0.856 

21.761 

72/73 

3.9% 

F/O 

ON i 

F- ‘ 

MVM-73 
(Part #2 
Only) 

JPL 

i 

C 1 

Boeing 

' ' ' 

1.075 

38.597 

. ■ ^ . i 

71/72 

2. 8% 

New: HI 


71-2 

AF V ^ 

LMSC 

1.378 

12.026 

70/71 

11.4% 

New: HI 


Average 

"""^Spaceeraft Contractor Program 
= High Inheritence 
(E) Estimated 


Fig. 6- 2 Costs of System Engineering 
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The definition of what that "something" is, is q-oite vague, and the associated 
cost accounting tracking for the task is even more vague and tortuous. 

0 NASA practices in structuring Complete WBSs,(vjhich have but a single 
place to charge each SB&I task) must be improved before accurate 
practice/cost evaluations can be made. 

6.6 Recommendations : Arising from these two major conclusions LMSC made 

the following, recommendations : 

o Discontinue . the SE&I analysis, until more, and better historical 
data become available. 

0 Develop a NASA-wide definition of SE&I functions, so that overlaps, 
such as those encountered with other functions such as subsystem eng- 
ineering, management, configuration management, and data management, 
can be eliminated, and better clarity can be obtained, 
o Develop a standardized ¥BS, to permit charging SE&I charges in a 
manner which ensures separation from charges relating to other 1 
functions; and also permits recording of cost variations arising 
from variations of SE&I practices from program-to-program. 
o Include these recommendations concerning the WS, and the NASA-wide 
definition of SE&I, together with specific instructions that they be 
used by future programs, in all futiure program RFPs. 


-X- As has been stated elsewhere in this report, the item highlighted by the ast 
erisk was recommended during the mid-term study review. The VffiS recommendation 
was authorised as, a new study task by NASA/HQ. 
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SECTION 7 


GROUND SUPPORT EQUIPJ-IENr (GSE) IMPACT ANALYSIS 

7.1. Analysis Objectives : This analysis, which formed, part 3 of Study Task 2 

had two major objectives. These were: 

(1) Defiriition of the approximate cost contribution of GSE to space- 
craft programs. 

(2) Identification of the cost impacting practices which lead to high 
er or lower GSE costs on a given unmanned spacecraft program. 

7.2. Analysis Approach ; The analysis included in its scope all program- 
peculiar GSE used at all locations (contractor and sub- Contractor plants, 

NASA and/or contractor field sites, and launch basej up to and including 
launch). Government furnished GSE was included in the analysis, but post- 
launch support eqiiipment used at the launch site, tracking stations (data 
links), and data reduction centers, were excluded. : - 

The Overall category of GSE was then sub-divided into several categories 
such as: 

0 Test & Checkout equipment r 

o Transportation & Handling equipment 
0 Servicing equipment 
0 Launch Control/Monitoring equipment. 

These catgories were further divided into equipment groups by type.; i.e. 
Test & Checkout equipment includes, manufacturing test equipment, simulators, 
test support equipment such as chambers & test fixtures, and integrated space 
craft, and systems test equipment. 

With the equipment categorized, the next step was to undertake three sub- 
tasks as follows: ; . / 

(1) Determination of type, quantity, and usage (location,, transfers 

& redundancies) of GSE used on historical and current spacecraft 
programs . u , 

(2) Tabulate historical costs of GSE, by major categories, and ' 
individual equipment items where possible. 

(3) Analyse the cost impact of specific GSE practices , 

.. - 
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Sub t-ask 3 included consideration of the following GSE practices : 
o Use of conmiercially available components 

0 Specifications imposed by the customer and the contractors 
o Eegree of standardization, within a program & among NASA centers 
0 Usage of common equipment for launch vehicle & payload 
o Scheduling of use for the same GSE item (geographical transfer 
vs item red\indancy) 

Finally, the average cost reduction was estimated for alternate low cost 
practices as synthetically applied to current unmanned space program hardware, 
and modularized future spacecraft with refurbishable /reusable components. 

7.2.1. Analysis Data Sources ; . Data were obtained from a number of programmatic 
sources as illustrated 'by Fig . 7.1 Page 7~3 » may be seen from the figure 
the data were fragmentary, sparse, and of a -"Total program, general lump sum 
type'.' Very little cost detail was available from these programs. 

A GSE Questionnaire was prepared for distribution to NASA centers, and contract- 
ors, but this was oriented toward GSE practices, father than the collection of 
detailed costs, and although it did furnish cost insights for 6 programs, 
actual cost fine detail was once more conspicuous by its absence. The data 
lack was fixrther complicated by a lack of coKimonality in GSE categories, des- 
criptions, and cost groupings, program-to-program. 

7.2.2 . Questionnaire- Outcomes ; Responses to the 3^4 page multiple -choice type 
questionnaire were received from one NASA center, and six (6) spacecraft prog- 
ram contractors. Highlights of thse responses were as follows: 

0 On a nine spacecraft program basis, contractor supplied GSE averaged 
4.7/ of TPC. Questionnaire results based on 6 programs revealed the 
same figure, although the detailed responses from programs showed 
contractor supplied GSE at 5.3^ TPC. 
o Based on the detail supplied by four (ij) programs, the GSS/GPE . 
breatcdown is as follows: 

o Contractor supplied GSE. . . 8Li$ . 

0 ; Govt Furnished GSE (GFE) .... 3.6/ 

The figures, listed do not inolude existing GSE in possession of the; contractor:, 
only that boioght new for support to the programs considered. 


EEPRODUCBILITY OF TEB 

Snal page is POOE 
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Fig. 7*1 GSE Data Sources and Availability 











































































































Figure T .2 furnished on Page 7-$ shows a Cost Exirapolation of the average 
costs of GSE, considering only the major categories of equipment. The values 
shown are the break-out cost details tabulated under the outcomes paragraph. 

7. 2. 2.1. Contractor GSB Cost Break-Down ; From the outcomes of the questionn- 
aire it was possible to gain , some insight into the manner in which overall 
GSE costs are incurred by contractors. While these values are necessarily only 
average values for the relatively few programs examined, they represent the 
best data available cuirrently on this subject. , 

Contractor GSE Cost Breakdown 
■ . (Percent) 

% Range Ave. % 


Planning and Requirements 

3-5 

4.4 

Design Engineering and De velopment 

10-40 

22,8 

Sustaining Engineering 

0-5 

1.4 

Procurement/Purchase/Lease 

10-50 

32.6 

Manufacture of New GSE 

10-30 

16. 5 

Modification of Existing GSE 

3-5 

4.3 

Qualification testing 

0-5 

1. 0 

Acceptance testing 

0-5 

1.4- 

Installation and Checkout 

0-5 

2.2 

Maintenance 

0-10 

3.2 

Documentation 

5-20 

10-2 

Total GSE 


TOO. 0% 


Source: GSE questionnaire, 5 S/C programs, 

7.2.2.2. Also from the questionnaire, and confirmed by NASA supplied data, it . 
was. possible to make a cost compariso3i among the several items of GSE supplied 
by the contfactors involved in .8 spacecraft programs, This comparison is tabul-' 
at ed in Fig 7.3;), furnished on Page 7-6. 
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MAJOR CATEGORY 

Contractor GSE 

GFE GSE I Total GSE Cost j 

% 

Extrap. $ M 

% 

Extrap. $ M 

$ 

Extrap. $M 

HANDLING & TRANSPORT EQ. 

12. 6 

0.33 

. 2. 0 

O 

o 

11.0 

0. 34 

SERVICING EQ. 

7.6 

0.20 

2. 0 

0.01 

6.8 

0. 21 

TEST & CHECKOUT EQ. 

79.8 

2. 07 

■ 

96. 0 

0.48 

82. 2 

2. 55 

TOTAL 

100. 0% 

• ?2.60M 

100. 0% 

■ $0. 50M 

100. 0% 

$3. lOM 

percent of total 

PROGRAM COST 

5.3% 

1.0% 


SILvRE OF TOTAL GSE 

84%. 

16% ■ 

100% 


* Based on four spacecraft programs. Excludes all contractor and GFE GSE costs which are 
sunk or capitalized by contractor. 


Fig. 7*2 Average GSE Costs Extrapolated b}'' Major Category * 
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Hughes 

$1.05^^^ 

Fairchild 

10.1 <«> 

Philco 

1. 985 
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.729 

Boeing 

2.067^^^ 

LMSC 

.921 

:TRW : 

4.3 

TRW 

2. 5 Est. 


Total 

Program 


Av. 

Year 

Expend. 


GSE 

%of 

T. P. C. 




110.3 



38. 597 


71/72 


71/72 


71/72 


72/73 


71/72 


70/71 


50. 0 


74/77 


(1) Excluding test equipment for experiments 

(2) Excluding some subsystem support equipment and system test equipment 

(3) Spacecraft contractor lii’ogram 
(E) Estimated 

(4) Excluding systems test complex 

(5) Excluding tracking station equipment ( $810K) 


Hg. 6 - Comparative Costs of Contractor Supplied GSE (as charged to the program) 
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7.3-» Major High, and Low Cost Practices; From an examination of the avail- 
able data it would appear that there is a major High Cost Practice associated 
with the programma.tlc use of GSE. This practice may be generalized as: 

The change in GSE req'uirements due to changes in the spacecraft^ 
and to a lesser degree^ due to some changes which occur in prog- 
ram planning and spacecraft eiqjeriment/payload changes. , 

In the area of Major Low Cost Practices , the following practice VJas observed: 

Extensive GSE inheritance is applied to spacecraft programs by. the 
contractors concerned. For the programs examined, less than of 
the total GSE value is charged to the program by any given contractor. 

A highly subjective breakdown of this Low Cost Practices is furnished below: 


Contractor supplied GSE charged to program 44% 

Contractor supplied GSE not charged to program 16% 

GEE GSE new or modified 9% 

GEE GSE sunk cost 30% 

Surplus GSE at replacement value i% 

Total EOM GSE Cost 100% 

(excl. post-launch GSE) 


7.U* Conclusions : Three major conclusions were drawn from the GSE analysis task; 

1. There is a definite trend toward low cost practices in the GSE 
area, evidenced by the high inheritance of GSE, program to program. 

2. Signifieimt cost sa-^/ings can be made in this programmatic area 

if better information on what exists, where, and in what quantities 
is disseminated to NASA contractors and NAS.A field canters. 

NASA is approaching this data dissemination task by preparing a 
GSE Equipment Visibility System (EVS). Further, as this listing 
. is available, RFPs will reo^uire the use of the GSE inventory, 
and require trade-offs of cost in cases of deviations . 

3- Maximum use of GSE exa.sting, is expected to reduce the programmatic 
costs approx. 
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SECTION 8 


NEW-LOOK STAl\roARDIZED TOS 

8.1, Task Objectives ; As initially planned, the objectives of this task were 
to generate : 

(a) A candidate "Nev7-Look" WBS. 

(b) A Users' Handbook. 

The purpose of these documents was to afford better visibility into the work 
performed on NASA programs by contractors, and NASA itself, and, in the case 
of the Handbook, to provide NASA with the capability to implement the new WBS 
on future spacecraft programs. 

Following the NASA/DVISC review of the preliminary concept of the scope of the 
task, early investigation revealed that the objectives as stated above, would 
be unattainable within the constraints of schedule and budget available for this 
brief study period. LMSC therefore proposed ths3 following modified objectives 
for the task, and obtained NASA concurrence to proceed with their implementat- 
ion: 

(a) Analyse and document the merits and difficulties associated 
with the development, and implementation of a standardized 
i^TBS. 

(b) Establish a means of tracking/recording data suitable for the 
evaluation of the cost impacts of Low Cost Practices, with any 
type of WBS. 

8.2. Analysis Background : Throughout the course of the Low Cost Program 

Practices study, one of the major aims has been the identification, and . 
quantification of High Cost Practices, applied to both past, and on-going, 

NASA spacecraft programs of the unmanned type , Many High Cost Practices have 
been identified successfully, but many others are suspected to exist, and, 
these have not proved quantifiable from historical programmatic records. IMSG 
sampled cost, and other programmatic data from thirty-nine (39) MSA and DoD 
programs, and in the process of analysis of these data, it became evident that 
the data were inadequate to permit quantification of program practices. Further, 
verification of data inadequacy was revealed.by . .a survey of more than seventy 
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(70) ftinctional managers and cost analysts within NASA, and the aerospace 
industry. Of those personnel surveyed, fifty-seven percent (5?/^) were of 
the opinion that data available were inadequate for quantifying impacts of 
program practices, nineteen percent (19^) believed that the data were adequate, 
and the balance expressed no opinion. 

Some NASa practices related to the implementation of program WBSs contributed 
directly to the the lack of adequate cost records, and it became evident that 
traditional practices preclude the quantification of either High or Low Cost- 
Driving Practices, in the several areas of: 

0 Systems Engineering and Integration 
o Flight Support Bqvd-pment 
o Ground Support Equipment. 

The scarcity of historical data, its adequacy when available, and the fact that 
NASA is in the process of conversion to more cost effective methods of conduct- 
ing near-term, and future space programs, gave rise to the conclusion that work 
should begin imraediately on the development of a I'JEVJ-Look IIBS. This programmat- 
ic work/cost analysis tool, together with its associated Implementation Direct- 
ives for Low Cost Practices, will be of material benefit in the prosecution of 
NASA policies of cost effectiveness for all future programs. 

8. 2»1 Analy sis Pr ob le m s : Several problems were encountered as this analysis 
developed. These problems are listed briefly below, as they were inshriomental 
in the decision to modiiy the scope of the \LBS analysis task from that origin- 
ally planned. 

(1) Unavailability of personnel . Due to the pressures of continuing the 
analyses undertaken in the areas of specifications, SE&I, GSE, and 
the documenting of the PER Survey results, work was deliberately 
delayed until the last quarter of the study. At that time, personnel 
originally scheduled to perform the analysis were unavailable. A 
contributory factor in the delay of the task, was an injxiry accident 
to the study Program Manager, under whose super''/ision the work was 
to be performed. 

(2) Underestimation of Task Scope : At the outset of the analysis, it was 
believed that, the LMSC version of the GSGSG technique, GASPER, a 
computerized cost acquisition and tracking program validated by the 
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' nj 

I i 

■ ■ H 

},[ 

h; 

t j 

USAFj would be acceptable for use with the evol'vlng WBS. Work was li 

- if 

commenced on the preparation of work packages for both program il 

functional tasks, and hardware down to the level of components. As 

this task progressed, and the level of detail proliferated, as well J| 

I I 

as. the number of inter work package functional relationships it be- tj 

} ? 

came obvious that the overall process of computation, tracking, and 1/ 

/reporting of all program effort costs was combinatorial, in nature. ]| 

!l 

CASPER pro^Tides excellent cost acquisition, and summing capability ;j| 

is' 

for work packages of the type traditionally uised with operations, lij 

. . U' 

but these are additive in nature, and unfortunatelv, combinatorial . [| 

. .. — y 

routines are not implicit, in the program. The program has . almost in- . : I 

finite capacity for handling discrete data, but when all possible if 

permutations of the many combinations of functional tasks, and the •/, 

hardware/software production* costs were considered, an unmanageable 
number, or more than 50,000 cost accounts resulted. The human effort f 

involved in manuall;/ recording such a large num.erical population, and / 

. ' fi 

accurately segregating one from another, is costly not only in time ‘ f- 

expended, but also in terms of the high probability of error. .fi 

(3) Funds were not available, nor would the study schedule permit re- . | 

programming CASPER to handle combinatorial routines. : 5 

8.3 Summary of Outc ome s : An initial concept was developed on the basis that any f f) 
MSl'^f-LOOK Standardized WBS would, of necessity, apply to not only work at aero- 
space contractors; but also throughout, all NASA centers involved in spacecraft 
programs. This initial concept is shown diagrammatically in Fig. 8.1 Page 8-l| . f ' h* 
Continuing work along the. lines shovni in the figusrs was infeasible for the raa- . / / 

sons stated above, and thus the modified task objectives were substituted. 

Outcomes of these are as follows: ; 

0 Lt'ISC did develop a Standardised V/BS concept for unmanned spacecraft f 

prograras, which, when fully developed, would be adequate for NASA f; 

requirem.ents. 

o Also developed were preliminary methods for recording Low Cost 

Program Practices used in conjunction with detailed program work- t 
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-packages, and to surffinarise work package quantified data. Both j.| 

of these work package oriented concepts are essential for the U 

evaluation of, and the quantification of, the cost impacts of l.j 

Low Cost PrograiR Practices. The concepts can be used, with, or it 

without a Standardized " 

8 . 1 i. Task/Analysis Conclusions: Among a number of conclusions to be drawn 

from the work performed on this Study Task, there are several of significance. ’il 

These are listed as follows: ;i 

0 EMSC under -estimated the task magnitude implicit in the devsl- y 

opment of a Standardized VJBS, A complicating factor in the ':J 

task was the attempt to incorporate requirements for tracking/ / 

recording details which would permit quantification of Low U 

Cost Program Practices. 

0 Both a verified initial concept, and a firm basis for the fj 

development at some future date of a Standardized WES were 
outcomes of the task as conducted to-date. ' - 

0 The concept established for standardization will provide adequate 
flexibility, within the work-package framework en''/isioned, to 
accommodate the small differences expected to exist among future 
unmanned spacecraft prograjns. 

o Traditional MSA practice has been to spend several millions of 
dollars per annum, in the tailoring of a new family of WBSs for 
each new program undertaken.. Implementation o.f a Standard 
ooul,d save this appreciable sum of money . for use elsewhere on 
unmanned spacecraft programs. 

0 Implementation of the concepts defined in this study, not only 
pro^/ide improved program ■'/isibility into cost/performance j but 
also furnish additional records for better evaluation of program 
practices. Further, there will be no disturbance of existing 
MSA. cost collection, and repo.rting systems, 
o Cost effective development of a; fully functional Standard VIBS viill 
require active participation of NASA personnel on a day-to-day 
basis. This will assist materially in the formulation, and co- 
ordination of intra-mural decisions, and definitions. 
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8.^. Task Recommendations ; LMSG recoirmiends that NASA; 

□Continue efforts to develop and subsequently implement a 
standardised family of VJBSs for unmanned spacecraft programs « 
o Establish a small team of NASA and contractor personnel to 
pursue full-time development of an integrated family of Stand- 
ardized VJBSs, and the speedy implementation of such a faraily for 
all future effort on 'onmanned spacecraft programs^ both NASA and 
contractor work being covered by the faraily aforementioned. 

For those wishing to study the VJBS task in depth, the full text is contained 
in EM LCPP-I 7 , which is included in the Appendix to this technical report. 
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SECTION 9 


RISK IMPACT OF LOW COST PROGRAM PRACTICES 

9.1 Analysis Objectives ; This analysis has been undertaken as the fifth sub- 
task of Study Task 2. As planned originally, the task had two objectives: 

(1) To list the Low Cost Practices identified during the course 
of the study, and against each practice listed formulate 
qualitative assessments of their, impact upon programs. 

These qualitative assessments were to made in terms of 

.High, Medium, or Low Risk to Program Performance , Program Cost , 
'Program Schedule . Rationale for each judgement listed against each 
practice was to be appended, and the change in Risk Impact from 
the usual routinely applied high cost practices to the recommended 
Low Cost Practices was to, be assessed as Risk Increase , Risk 
Decrease , or No Change in Risk . 

(2) To list type of Programmatic data required for the assessment of 

risk on a quantified basis , and to provide a preliminary plan 
for the acquisition of such data . . , 

9.1.1. Task Objective Substitution : As the study progressed it became increas- 
ingly apparent that programmatic data of the types required for assessment of 
program risk quantj-tatively , simply did not exist. Acquistion could not: be 
undertalcen thei’e fore, nor could valid planning for this task objective. 

A substitution of objective was made, with NASA concurrence . Objective #2 
then became: 

0 State of the . Art (sot a) Survey, Quantification of Program Risk. 

9.2. Analysis Approach : As with all the analyses undertaken, the approach was 

to prepare Ef4s covering the salient points of the analysis. Two EMs were 
preparedi EiMs LCPP-1$ , and LCPP-19 . issued to NASA in draft form October 8/?5, 
and November 20/7n respectively. The forraer treated the. qualitative : aspects of 
program risk analysis, and the latter furnished the details of the SOTA. survey. 

9.2.1. Qualitative Assessment Approach : All of the Low Cost Practices identified 

to date were listed by categoric groupings. The categories chosen were the sev- 
eral areas of . spacecraft program activity, each of: which contained a number of 
Low Cost Practices. 91 
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The program activities categories are; 


Activity Area/Category Qty of Low Cost Practices 

7 

8 
7 
2 

9 

0 (reserved for later edit 'ns) 

5 

2 


RFPs.. ..... LCP #1“3* Use RFP to encourage Lower-Cost Response 

Eesign...». LCP #2-6 Higher Design Margins-Less Testing 
Program Management... LCP #6-2 New. set of Low Coot CERs 
Quality Assxirance . . . . LCP #7**5 Reduce quantity of Inspection- 

Points . 

Specifications. ....... LCP #9-5 Use of a Reduced "Standard" Spec.- 

List. : 

For reader convenience , and again selecting an example at random, a completed 
copy of t?ie standard t/ork/Data Sheet developed for use in qualifying the 
Low Cost Practices, is furnished on Page 9-3 as Fig .9 •! 

9.2.2, State -of -the- Art Survey, Quantification of Program Risk Approach: 

As the title implies, the approach to this analysis segment consisted of a 
survey of recent publications having direct bearing on the subject. Each 
listed reference was scrutinized to deteradne approach hypothesis, , assessment 
methodology, degree of mathematical rigor, and: applicability to the assessment 
of spacraft program risk . More than ItO publications were scrutinized, and their 
salient, points were digested and included in the EM,, together with graphs, 
statistical form.ulae, and major conclusions encerpted from the publications., 

, . 9-2 


RFPs 

Design 

Documentation 

Manufacturing 

Program Definition/Changes 

Program Management 

Quality Assurance 

Reliability/Maintain ' y/Saf ety 

Specifications 

Testing 

Selecting examples at random : 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 


Paga ^ o f ^ 

1. LGP No. ; 2. CATEGORY ; QUALITY ASSURANCE 

3. LCW COST PflACTICfi ; Use of Contractor Quality Manual - Low-Cost Version 

Request contractor to prepare and submit a "low-cost version of his Quality- 
Manual which features low-cost practices listed here and elsewhere and in- 
cludes a basic program Qualit-y Plan. 

3.2 Validate the new Manual and permit use in subsequent contractor proposals in 
lieu of submitting a new program-peculiar Quality Plan. Require that program, 
peculiar changes or deviations be submitted as an addendum to the Low Cost 
Quality Manual. • 


U. RESULT OF IMPLEMENTATION ; 

U.l Eliminates need to submit a new Quality Plan with each proposal (large degree 
of redundancy among various plans). 

4. 2 Use of a low-cost Quality Manual would be a strong forcing function toward re- 
duction of program costs. 

4. 3 Use of validated contractor Quality Manuals would accelerate implementation of 
low-cost practices. • 


5 . AREA AFFECTED & DEGREE OF COST IMPACT ; (High^ Moderate, Low) 


Engineering 11 

Testing 

Raliabili-fcy 

All Other 

L_j 

Manufactm-ingl""! 

HaterialEZI 

Qual.Assvir, Ol] 


n 

DELTA RISK (Increa 

se , No Change 

, Decrease ) 



Performance No Change 

Schedule No Change 

Cost No Change 


7. RATIONALE FOR RISK ASSESSMENT ; 

7.1 Many of the Quality Assurance functions can be reduced without degrading 
the product quality or performance. Also, each of the elements of a new 
low-cost Quality Manual would be reviewed carefully by NASA before valida- 
tion, lowering the probability that any unforeseen program risk would be 
incorporated. Therefore, no change in program risk will result 

Fig. 9.1. Sample Data Sheet 
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9.2. 2*1 SOTA Survey Outcome s/Conclusions : Several conclusions resulted from 

the survey. I-Jhile the full text descriptive of the survey is available in 
M LCPP-19 > included in the Appendix; the conclixsions are repeated hereiJi;, as 
a convenience to the reader. The conclusions are: 

1. Apparently^ there is no e:d-sting methodology directly applicable to 
the prediction of quantified risk to program performance and schedule . 

2. Three (3) sets of data are required to make statistica,lly valid assess- 
ments of program risk: 

o Reasonably complete design definition of proposed new program 
hardware . 

•0 Accurate estimates of the cost of the several, and various, elem- 
ents of the new program. Detailed estimates are required down thr 
ough the component level of the WBS structure. 

0 Historical programmatic data from noth "Normal", and "Low Cost" 
programs. As a minimum, these data should include; 

(a) Performance Requirem.ents , for vehicle, subsystem, and 
component levels. 

(b) Schedule Information , prepared preferrably in PERT, or 
VERT type networks . 

(c) Cost Data/Estimates, at vehicle, subsystem, and component 
levels of detail, 

3. All of the foregoing data should be available on a, "standardized basis, 
so that summations, means, and standard deviations, can be determined. 
Such data would be used as risk bases for new prograns, so that compar- 
isons vJith historical program records can be imdertaken. 
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SECTION 10 


PROPOSED MSA POLICY (iMPLElffi^fTATIOW) DIRECTIVES 

10.1 The NASa/hQ Low Cost Systems Office expressed the intention to prepare j 
and issue a set of policy directives^ probably on an incrementel basis, to 
facilitate the implementation of Low Cost Practices for NASA spacecraft prog- 
rams. In compliance with this direction to prepare preliminary drafts of 
such directives, LMSC undertook to examine selected Low Cost Practices, and 
formulate them as potential NASA policies. The definitions, rationale, 
subject matter, and outcomes of this task, Stud 7 /' Task 3 -.were documented in 
EM LGPP-lS ' i5su.ed November 5 ^ 197$ • Appended to this EM were five ($) 
attachments, A,B,C,D,&E covering respectively: 

0 Basic Policy for Implementation of Low Cost Practices, 
o Basic Policy for Low Cost Application of Specifications & Standards 
o Basic Policy for Lower Cost Documentation Requirements 
o Basic- Policy for Improved Contract Program Definition, and Fewer 
Contract Changes. 

0 Basic Policy for Lower-Cost-Driving RFPs 

As there were no technological innovations in these directives rough draits, 
and as further, these drafts were intended to act only as basic guides to NA.SA 
in formulating the final directives, neither the EM, nor the attachments to it 
are furnished in either the Appendix, or this report. Copies of the were fur 
nished to NASA/HQ LCSO as per COR direction. 
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